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Preface 


This is the third report from an evaluation of the Principal Pipeline Initiative. The Wallace Foundation is 
sponsor of the initiative and of the evaluation, which is conducted by Policy Studies Associates, Inc., and 
RAND. This report analyzes progress in the initiative’s implementation in the six participating school dis- 
tricts from grant award in August 2011 through mid-2014. 
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Summary 


S ix urban school districts received support from The Wallace Foundation to address the critical chal- 
lenge of supplying schools with effective principals. The experiences of these districts may point the 
way to steps other districts might take toward this same goal. Since 2011, the districts have partici- 
pated in the Principal Pipeline Initiative, which set forth a comprehensive strategy for strengthening school 
leadership in four interrelated domains of district policy and practice: 

1. Leader standards to which sites align job descriptions, preparation, selection, evaluation, and support. 

2. Preservice preparation that includes selective admissions to high-quality programs. 


3. Selective hiring, and placement based on a match between the candidate and the school. 

4. On-the-job evaluation and support addressing the capacity to improve teaching and learning, with 
support focused on needs identified by evaluation. 

The initiative also brought the expectation that district policies and practices related to school leaders 
would build the district’s capacity to advance its educational priorities. 


The foundation selected the six sites, in part because they had already launched some policies and practices 
consistent with these components, and awarded grants of $7.5 million to $12.5 million over five years. The 
initiative also provided technical assistance. The districts agreed to adopt and implement approaches to 
standards-based preparation, selection, evaluation, and support for school leaders consistent with the ini- 
tiative’s specifications. The districts are: 



Charlotte-Mecklenburg Schools, North Carolina 


Denver Public Schools, Colorado 


Gwinnett County Public Schools, Georgia 
Hillsborough County Public Schools, Florida 
New York City Department of Education, New York 
Prince George’s County Public Schools, Maryland 

The evaluation of the Principal Pipeline Initiative has a dual purpose: to analyze the processes of imple- 
menting the required components in the participating districts from 2011 through 2015; and then to assess 
the results achieved in schools led by principals whose experiences in standards-based preparation, hiring, 
evaluation, and support have been consistent with the initiative’s requirements. This report addresses imple- 
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mentation of all components of the initiative as of 2014, viewing implementation in the context of districts’ 
aims, constraints, and capacity. 

Data collection for this report took place during the first three years after the initiative’s August 2011 
launch. Data sources were: ( 1 ) semi-structured interviews with administrators in district central offices and 
partners during site visits in each spring, 2012 through 2014; (2) focus groups with novice principals and 
assistant principals (i.e., those in their first, second, and third years in their position) in spring 2013 and 
2014; (3) surveys of novice principals and assistant principals in spring 2013 and spring 2014; (4) docu- 
ments including the districts’ proposals, work plans, and progress reports; and (5) observation of and par- 
ticipation in cross-site meetings from 2011 through 2014. 

Looking at the Whole Leadership Pipeline 

As the Principal Pipeline Initiative matured over three years, districts were rapidly developing the capacity 
to take a strategic look at the flow of talent into the principalship. The increasing availability of data 
opened the way to more active management of career progressions for aspiring leaders, using approaches 
that we describe in the subsequent chapters of this report. In particular, the Leader Tracking Systems re- 
quired and supported by the Principal Pipeline Initiative could inform this kind of active management. 



Although not all districts had fully developed datasets with which they could analyze the pathways taken 
by their new principals, all district leaders were well aware that positions as assistant principals were part 
of the usual sequence. Their scrutiny of that position and their consideration of ways to raise the bar for as- 
sistant principal performance showed the strategic thinking that they were bringing to bear. Our survey 
data suggest that they could also look strategically at the responsibilities assigned to other leadership posi- 
tions within schools, which may also provide important preparation for the principals who will take office 
some years in the future. 


Leader Standards 


A mandate to develop standards would not necessarily lead to active, continuing use of the standards, but 
in this initiative that is what ensued. Each district engaged in a multi-year process of developing and then 
refining leader standards — statements of the competencies and performance expected of principals. They 
developed and adapted indicators of competency and performance appropriate to various career stages, 
such as the stage of application to a preparation program or that of first-year principalship. Rather than 
leave their standards and competencies entombed in binders on their shelves, these districts’ leaders put 
them to active use in structuring principal preparation, hiring, evaluation, and support. They treated the 
standards as living documents, and they expected to make further revisions. 

Preservice Leader Preparation 

The districts took strong roles in reshaping preservice preparation. They developed or extended their own 
in-house preparation programs. They also forged stronger relationships and engaged in more active, de- 


manding communication with universities and alternate-route programs that prepared their principals. 

They worked with these partner programs to collect and review data on program features and outcomes. 

Principals newly hired in the three years 2011-12, 2012-13, and 2013-14 reported that they were well pre- 
pared for their responsibilities in such key areas as instructional leadership. These principals had been pre- 
pared through universities, alternate-route programs, and district offerings. Overall, their survey responses 
were fairly consistent with those of graduates of exemplary programs surveyed by Darling-Hammond and 
colleagues (2007). 

Selective Hiring and Placement 

During the grant period, all six districts revised the hiring process with further development and specifica- 
tion of hiring stages, criteria, and roles. They added new stages of assessment and selection for aspiring 
leaders well before the actual hiring stage. The results were hiring processes that were more systematic, de- 
liberate, and standards-based: 

Every district had a new or modified screening step producing a pool of candidates who were al- 
lowed to apply for leadership positions. 

Every district developed or modified selection criteria or instruments aligned with standards. 

Every district was using data on candidates and schools in more systematic ways to match candi- 
dates to specific school positions where district leaders believed they would be the best fit. 

Two districts, Denver and Hillsborough County, began conducting district-wide succession planning, using 
their improved data on potential principal vacancies to inform the overall hiring process. Finally, there was 
a noticeable sense of excitement in the districts around what officials saw as improvements in the hiring 
process. They were pleased with their use of added selection stages and assessments, and they expected to 
do more when they had easier access to still more data. 

Evaluation and Support 

Districts added capacity for evaluation and support by hiring and training additional principal supervisors 
and mentors or coaches. They built and at least piloted new evaluation criteria aligned with their leader 
standards, a process that required multiple rounds of clarifying the criteria and preparing supervisors to use 
them. As of 2014, they were working to develop the skills of both supervisors and coaches in supporting 
leaders, and they were planning to use data on individual leaders’ strengths and weaknesses to support 
greater differentiation of professional learning. 
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Conclusions 


Across components, district leaders were thinking strategically and seeing the feasibility of making long- 
term improvements in their principal corps through new policies and practices and the development of new 
capacity. They were especially enthusiastic about using new longitudinal data about principals and aspiring 
principals to inform both individual-level staffing decisions and also organizational decisions about 
strengthening leader preparation and support. The Leader Tracking Systems that they were building as part 
of the Principal Pipeline Initiative were expected to play an important role as a source of data for these de- 
cisions and were seen as an especially important addition to district capacity. 


District leaders also saw challenges ahead. In particular, they saw a need to redesign the assistant principal- 
ship as a learning opportunity for future principals. As of 2014 they had not made detailed plans in this re- 
gard, but they recognized the potential value of doing so. 

Future Evaluation Reporting 



This report is not the last word on the Principal Pipeline Initiative. Future reports on implementation and 
on effects will assess the actions taken by districts and their partner institutions in this initiative. A special- 
focus implementation report in 2015 will analyze principal evaluation systems. A 2016 report will assess 
implementation of the initiative overall, identifying the structures and policies put in place, the results ob- 
served, and factors that have supported or impeded the sites’ progress in carrying out their plans. After the 
end of the sites’ grant period, the final report from the evaluation will analyze the effects of “pipeline prin- 
cipals” — those prepared, selected, and supported in ways consistent with the initiative’s design — on impor- 
tant school outcomes, including growth in student achievement. 


1. Introduction 


S ix urban school districts are taking on the critical challenge of supplying their schools with effective 
principals, and their experiences may point the way to steps that other districts might take toward this 
same goal. With support from The Wallace Foundation, the districts have participated since 2011 in 
the Principal Pipeline Initiative. The initiative reflects a comprehensive strategy for strengthening school 
leadership in four interrelated domains of district policy and practice: 

1. Leader standards to which sites align job descriptions, preparation, selection, evaluation, and sup- 
port. 

2. Preservice preparation that includes selective admissions to high-quality programs. 

3. Selective hiring, and placement based on a match between the candidate and the school. 

4. On-the-job evaluation and support addressing the capacity to improve teaching and learning, with 
support focused on needs identified by evaluation. 

The foundation made a multi-year commitment to the six sites, which were selected in part because they 
had already launched some policies and practices consistent with these components. The districts receive 
grants of $7.5 million to $12.5 million over five years, along with technical assistance supported by the 
foundation, to expand and improve on their existing practices and to build toward a strong, sustainable 
policies and investments in school leadership. They agreed to adopt and implement approaches to stan- 
dards-based preparation, selection, evaluation, and support for school leaders consistent with the initia- 
tive’s specifications, which are described below. Each district also agreed that a large number of its new 
principals would experience these approaches within just a few years of the initiative’s start-up, by 2015. 
The districts are: 

Charlotte-Mecklenburg Schools, North Carolina 
Denver Public Schools, Colorado 
Gwinnett County Public Schools, Georgia 
Hillsborough County Public Schools, Florida 
New York City Department of Education, New York 
Prince George’s County Public Schools, Maryland 
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Background: Design of the Principal Pipeline Initiative 


After a decade of supporting research and program innovations in educational leadership, The Wallace 
Foundation saw compelling evidence that a principal has a pivotal role in a school’s instructional quality 
(see for example Seashore Louis, Leithwood, Wahlstrom, et ah, 2010) and that urban districts have the 
power to reshape principal preparation and support (Orr, King, & LaPointe, 2010). Now, in its Principal 
Pipeline Initiative, the foundation is testing the proposition that a school district’s programs, partnerships, 
policies, and practices can supply all or most of its schools with principals who are strong instructional 
leaders — strong enough to make a difference in school outcomes. The foundation’s 2011 request for pro- 
posals expressed the initiative’s central hypothesis in the following terms (Wallace Foundation, p. 2): 


If an urban district and its principal training programs provide large numbers of talented, aspir- 
ing principals with the right "pre-service" training and on-the-job evaluation and supports, the 
result will be a pipeline of principals able to improve teaching quality and student achievement 
district-wide, especially in schools with the greatest needs. 



The initiative reflects the belief that districts can do much to develop and support principals as effective in- 
structional leaders if they reshape traditional, often haphazard preparation and hiring processes for aspir- 
ing principals and if they restructure evaluation and support for new principals. The foundation intends to 
learn whether the participating urban districts can improve school leadership by means of an intentionally 
crafted system of clearly stated expectations, selection gateways, and opportunities for professional growth 
for new school leaders, implemented at scale. 


The Wallace Foundation’s request for proposals spelled out the four components of the pipeline to be im- 
plemented and tested in the districts: leader standards; high-quality preservice training; selective hiring and 
placement; and on-the-job evaluation and support. All four components were expected to work together. 
Preparation, hiring, evaluation, and support would all be aligned to the same standards and thus would be 
mutually reinforcing. Specifically, key elements of each component would be the following; 


STANDARDS for leadership would specify the capabilities and performance that the district 
wanted in principals, and the district would shape all its policies and practices related to aspiring 
and novice school leaders around these standards. In addition to using the standards as criteria for 
hiring principals, districts would ensure that preparation programs use the standards in admissions 
and in instruction; that the evaluation of new principals would assess the extent to which they meet 
each standard; and that on-the-job support would specifically address any gaps between standards 
and performance. 

PRESERVICE PREPARATION programs for aspiring leaders, whether lodged in universities, non- 
profit organizations, or districts themselves, were required to meet several specifications based on 
the findings of a major Wallace-supported study of exemplary preservice programs (Darling- 
Hammond, LaPointe, Meyerson, Orr, &c Cohen, 2007). Admissions would be selective, with appli- 


cants recruited and then screened according to the district’s standards for school leadership. The 
programs’ curricula, too, would align with district standards. In addition, the programs would or- 
ganize participants into cohort groups that would collaborate and progress together; link theory 
and practice through instructional approaches such as problem-based learning; and offer robust 
clinical experiences, expertly supervised. Drawing also on the findings of Orr and colleagues 
(2010), the initiative emphasized the importance of ongoing program improvement based on data 
about the quality of graduates’ preparation and, for programs based in universities or nonprofits, a 
partnership that would give the district a substantial role in shaping the program. 

In HIRING AND PLACEMENT, the initiative called for “rigorous processes of selection and 
matching.” Selection procedures would be designed to collect relevant information about appli- 
cants’ capabilities and to identify the strongest candidates by systematically assessing their capabili- 
ties against district standards. Districts would then assign new principals only to schools for which 
their particular strengths would be a good match. The initiative design further specified that dis- 
tricts should give preference in hiring to the graduates of those preparation programs that they 
judged to be of high quality. 


Finally, ON-THE-JOB EVALUATION AND SUPPORT for school leaders would also be aligned 
with standards. Districts would systematically evaluate novice principals’ instructional leadership 
capabilities, give them feedback, and, on the basis of the evaluation results, provide professional 
development and skilled mentoring to support them in further developing their needed capabilities 
The initiative defines novices as those in their first, second, and third years in the position. 



The initiative also brought the expectation that district policies and practices related to school leaders 
would build the district’s capacity to advance its educational priorities. All components would closely re- 
flect the district’s strategic priorities and would achieve district-wide scale and long-term sustainability. 
Building leadership capacity aligned to a larger agenda has become a goal of the initiative. 


From the beginning, The Wallace Foundation made several decisions that reflected its sense of urgency 
about testing the power of the pipeline to affect student performance. Because the aim was to measure out- 
comes within just a few years, and because it would take time for new cohorts of principals both to experi- 
ence the pipeline components and to make a difference in their schools, there was no time to waste on a 
slow start-up. In search of grantee districts that had a head start on implementation, the foundation invited 
applications from districts that enjoyed reputations for high-quality work on school leadership, and in the 
competition it awarded points for the “degree to which most of the components are already in place and of 
good to high quality” (Wallace Foundation, 2011, p. 8). It also made clear to grantee districts that contin- 
ued funding would depend on their success in placing substantial numbers of “pipeline principals” in 
schools by January 2015, less than three and a half years after grant award. With this deadline in mind, the 
foundation pressed for especially fast implementation of the initiative’s preservice preparation component 
so as to ensure that high-quality programs would be in place and producing many new principal candidates 
by that date. Changes in evaluation and support could be implemented at a slower pace if necessary, but 
would be in place by mid-2015. 
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Another important dimension of the initiative’s design was active partnership between grantees and The 
Wallace Foundation. This included not only each district’s ongoing dialogue with senior foundation offi- 
cials about detailed work plans and budgets, but also extensive opportunities to receive technical assistance 
from providers vetted by the foundation and to participate in the initiative’s Professional Learning 
Communities. The technical assistance included skilled help for each district in setting up its new data sys- 
tem, called a Leader Tracking System, as a repository for individual-level, longitudinal information about 
aspiring and novice leaders. The system would include data on the individual’s preparation and certification 
history, positions held and the schools in which he or she had held them, assessed skills and performance in 
each position, and participation in professional development and coaching. 

The initiative design addressed the ongoing improvement of pipeline components through data. For exam- 
ple, by using the Leader Tracking System to review aggregate data on the performance of the graduates of 
each preservice program, districts could identify the programs with which they wanted to continue close 
partnerships and those that needed improvement. This discerning approach to partnership with preparation 
programs was also to be supported by technical assistance in using Quality Measures, a suite of tools de- 
signed by Education Development Center with Wallace support, including indicators and rubrics for assess- 
ing leader preparation programs in a facilitated review of evidence. The foundation provided facilitation as 
part of its assistance with this initiative. 



This Report and the Overall Initiative Evaluation 

This report addresses implementation of all components of the initiative as of 2014, viewing implementa- 
tion in the context of districts’ aims, constraints, and capacity. Previous reports in this series described the 
districts’ plans and first-year activities (Turnbull, Riley, Arcaira, MacFarlane, & Anderson, 2013) and ana- 
lyzed the preparation and support for school leaders offered by districts and their partners (Turnbull, Riley, 
& MacFarlane, 2013). These reports and two future reports on implementation are part of a multi-year 
evaluation of the Principal Pipeline Initiative that has a dual purpose: to analyze the processes of imple- 
menting the initiative’s components in the participating districts from 2011 to 2015; and then to assess the 
results achieved in schools led by principals whose experiences in standards-based preparation, hiring, eval- 
uation, and support have been consistent with the initiative’s design. 


The evaluation team is issuing reports while districts are introducing changes in their policies and practices 
under the initiative, with the result that the findings in this and other implementation reports are prelimi- 
nary in several ways. District data on individual principals’ experience with the initiative (such as the num- 
bers of principals hired under particular procedures) are not yet complete but will become available for 
evaluators’ use beginning in 2015 through the Leader Tracking Systems. A separate study is beginning to 
analyze costs. 


This report is neither the first nor the last word on implementation of the Principal Pipeline Initiative. Data 
collection for this report took place during the first three years after the initiative’s August 2011 launch. 
Two years remain in the sites’ grant period, and implementation is a work in progress (Exhibit 1). In each 
year of the evaluation’s data collection, district leaders described policies and practices that they saw as 


consistent with the initiative’s intent, but in general not enough time had passed for them to offer more 
than preliminary impressions of the results of those policies and practices. Most district leaders spoke in in- 
terviews about key policies and practices they still intended to change in order to meet new or continuing 
challenges. The principals and assistant principals surveyed as part of the evaluation (as described below) 
might or might not have experienced the current policies and practices, depending on the career stage that 
each respondent had reached at the time a practice had been introduced. For example, a principal ap- 
pointed in 2013 might have completed formal preparation programs in 2010 and have been selected into a 
pool of principal candidates in 2012, before current practices went into effect. 


Exhibit 1: Timeline of the grants, the evaluation, and this report 


Aug 2011 

Grants 

begin 


Dec 2016 


2012 2013 

2014 

2015 

2016 

Grants 

end 



▼ 




Period described Grant activities continue 

in this report 


Nov 2011 

Evaluation 

begins 


III I II . Ill ^ ^ I I I 

2012 2013 2014 2015 2016 2017 


Apr 2018 

Evaluation 

ends 


Apr 2012-Aug 2014 — 

Data collection Evaluation of Evaluation of 

for this report implementation effects continues 

continues 



We offer this report’s findings as a picture of a substantial initiative, still under development in each site, 
that has already generated early lessons for the participating districts. In each chapter, “takeaways” capture 
district leaders’ experiences to date, which may provide food for thought for other districts. It is true that 
these six districts had considerable staff capabilities and also enjoyed funding and technical assistance from 
the Principal Pipeline Initiative. Still, their experiences and plans suggest strategies that other districts could 
explore, and their reflections suggest pitfalls and opportunities in implementation. 

Data Sources 

This report is based on an analysis of data collected by the evaluation team from the following sources: (1) 
semi-structured interviews with administrators in district central offices and partner organizations during 
site visits in spring 2012, 2013, and 2014 (Exhibit 2); (2) focus groups with novice principals and assistant 
principals (i.e., those in their first, second, and third years in their position) in spring 2013 and 2014; (3) 
surveys of novice principals and assistant principals in spring 2013 and spring 2014 (Exhibit 3); (4) docu- 
ments including the districts’ proposals, work plans, and progress reports; and (5) observation of and par- 
ticipation in cross-site meetings from 2011 through 2014, including observation of presentations and panel 
discussions by district and partner-program leaders. 
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Exhibit 2: District and partner interview respondents, by district, 2012-14 


District 

2012 

2013 

2014 

Charlotte-Mecklenburg 

14 

22 

11 

Denver 

19 

21 

13 

Gwinnett County 

14 

15 

16 

Hillsborough County 

17 

18 

30 

New York City 

8 

14 

18 

Prince George's County 

19 

23 

16 

Total 

91 

113 

104 


Exhibit reads: In Charlotte-Mecklenburg, 14 district or partner staff members were interviewed in 2012. 



Site-visit interviews were arranged by the project director in each district, responding to specifications from 
the evaluation team. We requested interviews with the project director, the superintendent (or, in New York 
City, another high-level official in the central office); other members of the executive team such as the direc- 
tors of human capital, curriculum and instruction, and data systems; and central-office staff and partner- 
program leaders who were, collectively, knowledgeable about standards, job descriptions, preparation 
programs, hiring and placement, supervision, evaluation, and support for principals. In making arrange- 
ments for the visit, each project director determined how many individuals could effectively inform the site- 
visiting team about each function without placing undue burden on district staff. Where two or three 
people worked closely together on a particular function, the team typically conducted a joint interview with 
them. In some cases project directors arranged one or more larger group interviews (e.g., the eight principal 
supervisors in Hillsborough County were interviewed in two groups in 2014, as were the eight principal 
coaches in that district). 


Interview questions were designed to elicit the respondents’ descriptions and assessments of each component 
of the Principal Pipeline Initiative in their local site as well as their overall perceptions of district policies and 
practices related to the principalship. A semi-structured protocol included not only factual questions but also 
probes for the respondents’ perceptions of what district practices appeared to be serving the intended pur- 
poses, what practices appeared to need improvement, and what changes (if any) were under consideration. 
The comments made by district staff and partners in the initiative’s twice-yearly gatherings also informed the 
evaluation team’s understanding and are the source of a few illustrative quotes in this report. 

Focus groups with novice principals and assistant principals were conducted in each district in 2013 and 
2014. Project directors identified the participants. The focus groups’ comments provided useful insights 
that informed the team’s understanding of the more systematically collected data, but because the partici- 
pants were not necessarily representative of their peers, caveats accompany the few focus-group comments 
referenced in this report. 




More complete data were gathered from novice school leaders in web-based surveys addressing perceptions 
and experiences related to preparation, hiring, evaluation, and support. The surveys were administered in 
spring 2013 and spring 2014 to all first-, second-, and third-year principals and assistant principals in the 
six districts. In the 2013 wave of surveys, a total of 353 principals responded, for an overall response rate 
of 57 percent (Exhibit 3). Among assistant principals, we report results from five districts, where the overall 
response rate was 82 percent among assistant principals in 2013. (The New York response rate was 17 per- 
cent among assistant principals in 2013, and we have excluded those respondents from our analysis.) A 
total of 541 principals responded in 2014, for an overall response rate of 66 percent. Among assistant prin- 
cipals, a total of 671 responded, for an overall response rate of 73 percent. 


Exhibit 3: Survey respondents and response rates, by district, 2013 and 2014 


2013 principal survey 

2013 assistant principal survey 

District 

Number of 
respondents 

Response rate, 
in percent 

Number of 
respondents 

Response rate, 
in percent 

Charlotte-Mecklenburg 

43 

96% 

62 

85% 

Denver 

56 

78 

91 

81 

Gwinnett County 

39 

93 

112 

85 

Hillsborough County 

55 

90 

47 

75 

New York City 

102 

31 

N/A 

N/A 

Prince George's County 

58 

89 

36 

78 

Total 

353 

57 

348 

82 


2014 principal survey 

2014 assistant principal survey 

District 

Number of 
respondents 

Response rate, 
in percent 

Number of 
respondents 

Response rate, 
in percent 

Charlotte-Mecklenburg 

52 

96% 

89 

84% 

Denver 

68 

83 

91 

84 

Gwinnett County 

47 

100 

146 

90 

Hillsborough County 

42 

89 

97 

82 

New York City 

269 

51 

208 

54 

Prince George's County 

63 

93 

40 

87 

Total 

541 

66 

671 

73 


Exhibit reads: In 201 3, 43 principals completed the survey in Charlotte-Mecklenburg, comprising 96 percent of all first-, second-, and third-year principals surveyed in the district. 
Note: 2013 NYC assistant principal data were not included in analyses due to a low response rate. 
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Data Analysis 

For this study of initiative implementation across six districts, the district is the primary unit of analysis. 
Therefore, where this report presents overall survey findings from all districts, the analysis gives equal 
weight to each district. The raw survey responses, if compiled across districts, would over-represent New 
York City — which had the largest numbers of novice principals and assistant principals — and under-repre- 
sent the other districts. Because of the significant cross-district differences in numbers of novice principals 
and assistant principals, we applied post-stratification weights so that each district would be equally repre- 
sented in overall analyses. Accordingly, overall cross-district findings are presented as weighted data. For a 
complete explanation of the survey weights, see Appendix A. 

In the analysis of survey data, due to the relatively low response rate from principals and assistant princi- 
pals in New York City (51 and 54 percent, respectively in 2014), we tested for non-response bias to deter- 
mine whether respondents were representative of all principals and assistant principals in the district. We 
used New York City’s progress reports submitted to The Wallace Foundation to determine whether there 
were significant differences between the percentages of survey respondents who reported having graduated 
from particular preparation programs versus the percentages of graduates of those programs reported in 
the more complete data from the district. Because three programs were overrepresented among survey re- 
spondents, additional analyses tested a sample of survey items for differences in responses from graduates 
of each of those programs versus graduates of all other programs. We found only a few significant differ- 
ences among group means, and these few significant differences did not follow a clear pattern or cluster 
within a particular set of items. We therefore determined that within-district survey weights were not neces- 
sary for New York City. 

Qualitative analysis was iterative. Interview transcripts and notes were coded by the evaluation team, using 
software for qualitative data analysis, according to key features and expectations of the Principal Pipeline 
Initiative. Multiple iterations of analysis identified and refined the specific themes, descriptions, and analy- 
ses presented in this report. Drafts were reviewed by the particular team members who visited the site for 
factual accuracy and revised as necessary. The districts’ progress reports to The Wallace Foundation were a 
supplementary source for detailed factual descriptions of policies enacted; similarly, their initial proposals 
were a source for facts about policies and practices in place before the initiative. Finally, project directors at 
the sites conducted a further fact-check prior to publication. 


Future Evaluation Reporting 

As the initiative and the evaluation continue, two additional reports on implementation and one on effects 
will assess the actions taken by districts and their partner institutions in this initiative. A special-focus im- 
plementation report in 2015 will analyze systems for evaluating principals’ performance. A 2016 report 
will assess overall implementation, identifying the structures and policies put in place, the results observed, 
and factors that have supported or impeded the sites’ progress in carrying out their plans. After the end of 
the sites’ grant period, in 2018, the final report from the evaluation will analyze the effects of “pipeline 
principals” — those prepared, selected, and supported in ways consistent with the initiative’s design — on im- 
portant school outcomes, including growth in student achievement. 



2. Looking at the Whole Leadership Pipeline 


he six districts selected for the Principal Pipeline 
Initiative had already launched policies and pro- 
grams in school leadership at the time of grant 
award, but under the initiative they have had further 
opportunities to deepen their strategic thinking. The 
districts have seized the responsibility — and opportu- 
nity — to reshape career pathways for aspiring and 
novice principals. Therefore, this chapter begins with an 
analysis of the problems in career advancement that dis- 
tricts wanted to solve when they embarked on the ini- 
tiative, then provides descriptive data on entry into 
principal positions over the life of the initiative to date. 
The data currently available for these analyses begin to 
provide a glimpse of the potential of human-capital 
data systems to inform district planning, a theme to 
which later chapters of this report return. This chapter 
concludes with an analysis of district leaders’ views on 
strategic management of their pipelines as of 2014. 


Takeaways 

1 . District leaders sought to strengthen the 
caliber of new principals and candidates for 
principal positions. 

2. Most novice principals spent time — several 
years, in some districts — as assistant 
principals. 

3. Most principals also served in other school 
leadership positions such as teacher leader, 
department chair, curriculum specialist, 
literacy or math coach, and others. 

4. Districts were reframing the expectations 
for specific positions that they knew their 
future principals were likely to hold, starting 
with the assistant principalship. 



Strengthening the Caliber of Leaders 

When the Principal Pipeline Initiative began, leaders in all of the districts reported that a central problem to 
be solved in school leadership was a haphazard pattern of career progression: individuals without notable 
leadership talents could acquire administrator certification, develop their networks, and win appointment 
to school leadership positions, while others with more potential might be overlooked or not even try to 
move up. Leaders in each district also said they had begun to address this problem on their own, but that 
the Principal Pipeline Initiative would accelerate their progress. 

District leaders wanted to improve the rate of success and retention among new principals by cultivating 
and selecting strong candidates. The “bench,” district leaders said, was not as deep as it needed to be. For 
example, some explained in interviews that they did not always see more than one strong candidate for 
principal positions. The number of applicants for vacancies was a particular concern in Denver and New 
York City, where strong accountability systems had resulted in principal dismissals, thus creating vacancies 
to fill, and where some interviewees expressed the view that the risk of dismissal might be making the job 
unattractive to applicants. In Hillsborough County, a shortage of very strong applicants was accompanied 
by an oversupply of applicants with the formal qualifications for principalship, making it hard to find an 
applicant whose success as a principal was most likely. Interviewees in Gwinnett County harked back to the 
previous decade when rapid growth in student enrollment had left the district short of well-qualified princi- 
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pal applicants. Charlotte-Mecklenburg leaders described their recent investments in leader preparation pro- 
grams and selection procedures, expressing optimism that they were beginning to address the problem. 

Leaders in all of the districts hoped that the initiative would result in a larger number of strong applicants for 
principal positions and a higher success rate among novice principals. They saw participation in this initiative 
as a chance to impose greater — and smarter — managerial control over the process by which individuals at- 
tained jobs in school leadership. District leaders wanted to manage career paths much more intentionally. 


Principal Hiring Rates 

A starting point for understanding the issues around aspiring and novice principals is the most basic data 
on the number of principals hired for each school year from 2011-12 through 2013-14. (Since the number 
of schools remained fairly stable in these districts, the number of newly hired principals was roughly equal 
to the number of individuals leaving the principalship.) In general, these six districts replaced roughly 15 
percent of their principal corps annually (Exhibit 4). Denver was an outlier, with a hiring rate that was con- 
siderably higher (26 percent) for 2011, but its rate fell to one similar to those of other districts over the fol- 
lowing two years. Prince George’s County was an outlier in the other direction for 2013-14, appointing 
new principals at only a 7 percent rate. 


Exhibit 4: New principals appointed, by district and school year, 
as a percentage of principal positions in the district 


District name and total 
number of principal positions, 
2013-14 

New principals as a percentage of all principal positions 
2011-12 2012-13 2013-14 

Charlotte-Mecklenburg (160) 

14 % 

12 % 

13 % 

Denver (182) 

26 

20 

16 

Gwinnett County (132) 

14 

13 

12 

Hillsborough County (254) 

13 

18 

14 

New York City (1,590) 

12 

13 

13 

Prince George's County (205) 

16 

18 

7 


Exhibit reads: Charlotte-Mecklenburg had 160 principals in 2013-14 and in 2011-12 appointed a number of new principals that was 14 percent of this total. 
Source: Districts’ annual progress reports to The Wallace Foundation. 


All the districts had a sense of their likely principal-replacement task at the start of the initiative, based on 
their recent experience, but their procedures for data tracking and projections varied. Gwinnett County 
made generally accurate projections based on factors such as historical experience, projected student enroll- 
ment, and the number of principals nearing retirement age. Hillsborough County had a considerable 
amount of data on each employee, including principals, and also worked to make accurate projections. In 





Denver, where accountability pressures were especially intense and human-capital data systems less fully 
developed, district officials knew that a high replacement rate was likely but did not have specific projec- 
tions. Over the course of the initiative, the foundation supported all districts’ investments in more system- 
atic, data-informed procedures for succession planning. District leaders expressed enthusiasm about this 
change, which we discuss throughout this report. 


Positions Held Before the Principalship 

Most new principals in these districts had been assistant principals, as shown by the survey data collected 
for this evaluation. Overall, among first-, second-, and third-year principals responding to the survey in 
2014, the percentage who had been assistant principals was 84 percent, weighted (Exhibit 5). 1 It was 87 
percent, not significantly different, in the previous year. Across districts, the median number of years spent 
as an assistant principal was five years according to both the 2013 and 2014 surveys. In 2014 it was high- 
est, six and a half years, in Hillsborough County (a district that required a lengthy apprenticeship for aspir- 
ing principals, as this report describes) and lowest, two years, in Denver (the district that was hiring 
principals at a rapid clip at the start of the initiative). 


Exhibit 5: Principals who had served as assistant principals and those 
who had also served as teacher leaders, by district, 201 4 
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■ Assistant principal 

■ Teacher leader and assistant principal 



CMS DPS GCPS HOPS NYC PGCPS OVERALL 

(N=52) (N=67) (N=47) (N=40) (N=237) (N=61) (Weighted N=522) 


Exhibit reads: Eighty-one percent of first-, second-, and third-year principals in Charlotte-Mecklenburg responding to the survey in 2014 had served as an 
assistant principal, and 60 percent had served as both a teacher leader and an assistant principal. 



Source: Principal Survey for “Evaluation of the Principal Pipeline Initiative,” 2014. 


1 As described in Chapter 1 and Appendix A, survey data drawn from all six districts are reported here as weighted frequencies, such that each district is equally 
represented in the overall number or percentage shown. District-specific results are reported unweighted. 
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Participants in our survey clearly recognized the learning opportunities that came with doing the job of an 
assistant principal. Among the principals who had served as assistant principals, 79 percent said that the 
experience contributed to their overall preparation for school leadership “to a great extent.” 2 

Beyond this widely shared experience as assistant principals, however, the novice principals reported a great 
range of leadership experiences. Among those who had served as assistant principals, most (89 percent) had 
held at least one other leadership position in a school as well. The pathways through school leadership po- 
sitions varied a great deal, and there were some systematic differences by district. For example, in Gwinnett 
County it was customary for aspiring principals to serve as both teacher leaders and assistant principals; 
nearly three-quarters of the novice principals (72 percent) had done so (Exhibit 5). Only half as many prin- 
cipals in New York City (36 percent) had held both of these positions, however. (We use the term “teacher 
leaders” to describe the response that appeared in the survey as “grade level or subject area team leader/ 
chairperson.”) 

We explored the particular roles that principals had held because survey respondents reported that their 
prior experience in “multiple roles” had contributed to their preparation for the principalship. Among all 
responding principals, 63 percent reported that experience in multiple roles had contributed to their prepa- 
ration “to a great extent,” and another 19 percent reported that it had done so “to a considerable extent.” 

Novice principals’ prior leadership positions were different from district to district. There were cross-dis- 
trict differences, for example, in principals’ past experience in school leadership positions that might call 
for instructional leadership — potentially signaling differences in new principals’ opportunities to prepare as 
instructional leaders. We surveyed principals on their experience as department head, curriculum specialist 
or coordinator, or literacy or math coach. With the last two groups combined into those who had been a 
“curriculum specialist and/or coach,” Exhibit 6 illustrates the ways in which positions in department or 
curriculum leadership, respectively, had been part of the principal pathways in each district. In 
Elillsborough County, neither of these roles was reported by more than 18 percent of respondents. Instead, 
it was the job of assistant principal that had provided most of the experience in leadership for future princi- 
pals. In New York City and Prince George’s County, however, both the role of department head and that of 
curriculum specialist or coach were substantial parts of the pathways to a principalship. In New York, cur- 
riculum positions were reported by just over half of novice principals (54 percent), and a subset of this 
group (17 percent of all the responding principals) had also been department heads. In Prince George’s 
County, 64 percent of principals had been department heads, 46 percent had held curriculum positions, and 
28 percent had served in both of these roles. Only 15 percent of Prince George’s County principals had 
been assistant principals without also having been department heads or curriculum leaders. 

The variation in pathways appears to reflect a combination of district policies and individual career trajec- 
tories. Gwinnett and Elillsborough Counties had a policy of drawing virtually all of their principals from 
the ranks of assistant principals. However, the extent to which aspiring leaders gained experience in other 
school leadership positions varied both within and across districts. No particular sequence of positions was 


2 This question did not have an exact counterpart in 2013. Throughout this report, 2014 results are reported alone where a question was newly added or revised in 
the 2014 version of the survey. 



Exhibit 6: School leadership positions previously held 
by principals, by district, 2014 


Charlotte-Mecklenburg (N=52) 



Curriculum 
specialist 
and/or coach 

38 % 


None of these 
positions 


Assistant principal 

81 % 


Department 

head 


25 % 


Denver (N=67) 



ant principal 

85 % 


Department 

head 

30 % 


None of these 
positions 


Curriculum 
specialist 
and/or coach 

39 % 


Gwinnett County (N=47) 

Assistant principal 

94 % 


Department 

head 

23 % 


Curriculum 
specialist 
and/or coach 

30 % 



None of these 
positions 


New York City (N=237) 



Curriculum 
specialist 
and/or coach 

54 % 


Assistant principal 

68 % 


Department 
head 

29 % 


None of these 
positions 


Hillsborough County (N=40) 


Curriculum 
specialist 
and/or coach 

18 % 



Assistant principal 

98 % 


Department 

head 


13 % 


None of these 
positions 


13 


Prince George’s County (N=61) 


Assistant principal 

77 % 



Exhibit reads: Eighty-one percent of first-, second-, and third-year principals in Charlotte-Mecklenburg had been an assistant principal, 38 percent a 
curriculum specialist and/or coach, and 25 percent a department head prior to becoming a principal. Forty-four percent of principals had been an assistant 
principal only, 10 percent a curriculum specialist and/or coach only, and 2 percent a department head only. Seventeen percent had been an assistant 
principal and curriculum specialist and/or coach, 12 percent an assistant principal and a department head, 4 percent a curriculum specialist and/or coach and 
a department head, and 8 percent had been all three. Four percent had held none of these positions. 

Source: Principal Survey for “Evaluation of the Principal Pipeline Initiative,” 2014. 
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prescribed; instead, individual aspiring principals found their way through a series of professional opportu- 
nities in which they could practice leadership skills. 

Having developed a Leader Tracking System, each district will be able by early 2015 to view individual 
and aggregate data on these pathways to the principalship. Increasingly, district leaders will be able to 
identify common sequences and, if they so choose, to reshape the sequences through policies and pro- 
grams. In addition, districts could redesign the leadership roles that are often stepping stones to a princi- 
palship in the district, ensuring that those roles offer experience and responsibility in critical areas like 
instructional leadership. 


Building a Career Progression Strategically 

Some district leaders volunteered a detailed picture of a leadership pathway in interviews that the evalua- 
tion team conducted in 2014. Two top leaders outlined in detail the stages of leadership opportunities and 
selection gates that they considered desirable. One spoke of an ideal progression into the principalship — 
one that did not yet exist in the district but that might be created through district policy: 

The first part is robust outreach that cultivates leadership. It is a wide net and never overlooks tal- 
ented people. Next is a very intentional pathway into positions that would lead to assistant princi- 
pal, [such as] a teacher leadership position like department chair or staff development [position in 
which they] assess the needs of the school and think like a principal. So after three years, they are 
good assistant principals; the third stage is the assistant principalship. Fourth is specifically posi- 
tions that lead to a principalship, an apprenticeship or internship we give you that accelerates you 
so you have experiences that get you prepared, trained, and ready. Next is the first principalship. 

A top leader in a different district made similar comments, emphasizing selectivity in the steps through 
which that district had begun to move aspiring principals forward as of 2014: 

We developed [standards for a] teacher leader, [identifying] what does excellence mean to the 
teacher leader. And partly that’s a feedback tool that’s very intentionally a selection tool to help our 
school leaders select good teacher leaders to get in the pipeline in the first place. So that’s selection 
option number one. We then have selection opportunities for [university] programs to become an 
assistant principal.... In order to become assistant principal in the district you have to get into a 
pool, so there's a rigorous selection. Then as you go up there’s selection, a very rigorous process to 
get into our principal residency program, so again, a nice selection tool. And then you have to get 
into the principal pool beyond that, and then there's a selection process [for principal placements]. 

Across all six districts, leaders spoke of both the opportunities and selection gates in place for aspiring lead- 
ers. As illustrated by the quotations above, some districts’ leaders saw the assistant principalship as an ex- 
pected part of the progression, a pivotal position from which the best performers could be selected for a 
final stage of preparation and then potentially qualify for selection as principals. This brought increasing 
recognition of some challenges, however, as we discuss next. 



Challenges Related to Assistant Principalships 


The challenge most visible to district leaders as they described career progressions as of 2014 was that of 
reframing the assistant principalship. They described their hopes, concerns, and possible policy options but 
had not finished developing specific plans for this reframing. Rather than letting principals shape the re- 
sponsibilities of their assistant principals, district leaders wanted to use the assistant principalship strategi- 
cally, as both apprenticeship and proving ground for future principals. Most of the assistant principals saw 
it this way as well: among first-, second-, and third-year assistant principals responding to our survey in 
2014, a weighted total of 84 percent either planned to apply for principal positions (61 percent) or had al- 
ready done so (23 percent). From both the central office and the assistant principal’s office, then, there was 
interest in structuring the assistant principalship to offer better learning opportunities. 

In all districts except Gwinnett County, however, district leaders expressed concern about the current assis- 
tant principals who had not shown high potential to meet the district’s expectations as principals. This was 
a sensitive topic for district leaders to discuss in interviews. They recognized that many assistant principals 
were capably carrying out the kinds of noninstructional roles that in past decades were at the core of 
school leadership, such as student discipline and building management, and they were careful to communi- 
cate respect for the work those assistant principals were doing. District leaders were beginning to strategize 
about career opportunities other than principalships that might appeal to these assistant principals, such as 
central-office positions in student services or other departments. Such exit strategies for sitting assistant 
principals could free up positions for new assistant principals who would bring not only ambition but also 
skills in instructional leadership. 

At least one leader in each of four districts 3 expressed impatience with another group of assistant princi- 
pals, those who were not performing well in their positions. One commented that principals were some- 
times too reluctant to give their assistant principal a poor rating. “Hard conversations” were needed, this 
interviewee said. A top leader in another district also expressed frustration, saying: 

We have assistant principals ... that are not keeping up with the changes. They’re satisfied with 
where they are, and they’re not adapting to the new instructional focus, . . . haven’t kept up with 
the educational changes that are taking place. 

When they thought about pathways to leadership, then, district leaders were planning possible trajectories 
for the people who would become principals and also exit strategies from school leadership for those who 
probably would not. They recognized a need to open up assistant principal vacancies for strong candidates, 
who would otherwise grow frustrated with the lack of opportunities. 


3 At the request of some of these interviewees, we do not name the districts here. 
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Summary: Changes under the Initiative , and a Look Ahead 


As the Principal Pipeline Initiative matured over three years, districts were rapidly developing the capacity 
to take a strategic look at the flow of talent into the principalship. The increasing availability of data 
opened the way to more active management of career progressions for aspiring leaders, using approaches 
that we describe in the subsequent chapters of this report. In particular, the Leader Tracking Systems re- 
quired and supported by the Principal Pipeline Initiative could inform this kind of active management. 

Although not all districts had fully developed datasets with which they could analyze the pathways taken 
by their new principals, all district leaders were well aware that positions as assistant principals were part 
of the usual sequence. Their scrutiny of that position and their consideration of ways to raise the bar for as- 
sistant principal performance showed the strategic thinking that they were bringing to bear. Our survey 
data suggest that they could also look strategically at the responsibilities assigned to other leadership posi- 
tions within schools, which may also provide important preparation for the principals who will take office 
some years in the future. 
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3. Standards for School Leaders 


I he Principal Pipeline Initiative called for develop- 
ing standards for school leaders as an early step 
in implementation. The intention was that the 
standards would codify expectations for school leaders’ 
capabilities and performance, and that they would shape 
leaders’ preparation, hiring, evaluation, and support. 

Over time, the districts’ initial standards documents 
were supplemented with companion documents laying 
out desired principal competencies and, often, rubrics 
and evidence guides, as described below. Thus the work 
of building out and revising standards took place over 
multiple years. We analyze here the ways in which dis- 
tricts used standards to drive policy and program 
changes, and what the early results have been. 

Standards Development 

At the beginning of the initiative, two purposes of stan- 
dards development stood out for district leaders. First, in all six districts, officials said that developing and 
using standards and competencies would lend coherence to the district’s leadership policies and practices. In 
Hillsborough County, for example, leader standards were called “the driving force,” and the development of 
standards was a matter of “putting first things first.” District leaders in all sites intended to align job descrip- 
tions and hiring criteria, preparation curricula, leader evaluation, and ongoing support to their standards. 


Takeaways 

1 . Districts treated their standards 
documents as living documents. 

2. Districts used the structure and language 
of standards and competencies to build 
preparation curricula, hiring criteria, and 
tools for principal evaluation and support. 

3. Standards and competencies were 
important influences on professional 
development, hiring, and evaluation. 

4. Standards and competencies were revised 
as needed improvements came to light in 
the process of using them. 

5. Districts expected to continue adjusting 
their standards documents. 
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Second, interviewees in three districts in 2012 also spoke of standards in a way that emphasized the power 
of standards to communicate priorities for school leadership. In Gwinnett County, one district leader said 
“the standards have to be the centerpiece, ... a way of communicating what you want to achieve,” while 
another said, when asked about standards and job descriptions, “We want all of us to speak the same lan- 
guage.” Similarly, in New York, a leader said, “We needed to create consistency of language around leader- 
ship; the [previous] school leadership competencies were not used in any systemic way.” Leaders in Prince 
George’s County said in 2012 that they were pleased with the opportunity to align their policies on school 
leadership to a set of standards. They also expressed the sense that a “definition around leadership that’s 
more clearly and concisely connected to all our work” was helping in communication. 

The districts convened groups of varying composition to launch their standards development. They had 
two starting points in common: where the state had articulated standards, districts incorporated those stan- 
dards into the process; districts also consulted the 2008 standards of the Interstate School Leaders 
Licensure Commission (ISLLC), a consensus document of many professional associations that had greatly 


Policy Studies Associates, Inc. 




Districts Taking Charge of the Principal Pipeline 


influenced state regulations nationwide. In most districts, as described below, principals were involved in 
the deliberations during 2011-12. District leaders explained in interviews that they expected principals’ par- 
ticipation in standards development to bring both practicality and credibility to the standards documents. 

Charlotte-Mecklenburg had a prescribed set of standards and evaluation requirements from the 
state of North Carolina, but nevertheless formed a School Leadership Council including principals, 
assistant principals, principal supervisors, human resources staff, preservice partners, and the chief 
academic officer to focus on standards. This council worked during 2011-12 to identify key com- 
petencies for leadership practice aligned with a subset of the state standards, which it called “Super 
Standards,” reflecting the district’s needs and priorities. 

Denver developed its own leader standards, as it had done for its parallel document, the 
Framework for Effective Teaching. A small committee spent time in summer 2011 in a “very, very 
labor intensive” process of developing the first draft of what was later named the School 
Leadership Framework, reflecting state standards, tools and competencies emphasized in a partner 
preservice program, and standards drawn from national sources. The draft was then extensively 
vetted by district stakeholders. 

Gwinnett County already had job descriptions aligned to Georgia’s leadership standards, but the 
district also asked outside experts to review its standards. 

Hillsborough County developed and refined a lengthy set of School Leader Standards and 
Competencies in a process that engaged a 20-person committee of principals and assistant princi- 
pals, then asked all principals and assistant principals to vet the resulting competencies. State stan- 
dards were incorporated. 

New York convened a cross-division group to develop a vision for a career continuum of leader- 
ship competencies. The group set out to align the competencies to the Quality Review — the school 
observation and evaluation process that uses a rubric to assess how well the school is organized to 
support student achievement — as discussed in detail later in this chapter. 

Prince George’s County drafted a set of standards for vetting internally, by stakeholder groups that in- 
cluded the teachers’ and administrators’ unions, and externally, by the state department of education. 

Four districts saw their initial standards documents as unwieldy, recognizing that more concise standards 
and competencies would be more understandable and more usable in district practice. Denver and 
Gwinnett County saw that the early versions of their documents were repetitive and needed to be pared 
down. Hillsborough County focused its attention on eight competencies, selected from a longer array that 
was initially developed. Charlotte-Mecklenburg updated its standards in 2014, renaming them the 
Competencies, Skills and Beliefs for School Leaders, around which they aligned district policies on prepara- 
tion, hiring, evaluation, and support. The changes have continued, as this chapter describes. One district’s 
recently revised standards are shown for illustrative purposes in the text box below. 



An Illustration of School Leadership Standards 

As of 2014, each of the six districts had from six to ten overarching principal standards. Most of the standards documents 
specified skills, elements, or indicators providing detail or operationalizing the overarching standards. To illustrate, we 
show Denver's principal and assistant principal standards (called "expectations") and indicators, as revised in 2013. 


School Leadership Framework: Denver Public Schools 


Expectation 

Indicator 

Culture and 

Leads for equity toward college and career readiness 

Equity Leadership 

Leads for culture of empowerment, continuous improvement, and celebration 

Instructional 

Leadership 

Leads for high-quality, data-driven instruction by building the capacity of teachers to lead and 
perfect their craft 

Leads for the academic and social-emotional success of diverse student populations 


(ELA Program School Leaders): Leads for effective English Language Acquisition programming 

Human Resource 
Leadership 

Applies teacher and staff performance management systems in a way that ensures 
a culture of continuous improvement, support, and accountability 

Implements strong systems for identifying, recognizing, and distributing talent 

Strategic 

Leadership 

Leads the school's vision, mission, and strategic goals to support college readiness for all students 


Distributes leadership to inspire change in support of an empowered school culture 

Organizational 

Leadership 

Strategically aligns people, time, and money to drive student achievement 

Community 

Leadership 

Actively advocates for members of the school community and effectively engages family and 
community 


Using Standards in District Policy and Practice 

After formulating their initial leader standards and competencies in 2011-12, the districts embarked on the 
longer process of aligning components of the pipeline to those documents. As districts revised or developed 
hiring systems, they were able to build from a foundation of newly adopted standards. In Charlotte- 
Mecklenburg, for example, the identification of Super Standards was followed first by building an evalua- 
tion instrument and procedures designed to identify an individual leader’s strengths and weaknesses, then 
by refining the principal job description so that it reflected both the evaluation instrument and the stan- 
dards, and then by repeatedly fine-tuning recruitment and selection processes to improve their alignment 
with the job description. 
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In Hillsborough County, a person closely involved with the ongoing use of standards and competencies de- 
scribed their centrality in the district’s work with principals, assistant principals (APs), and aspiring leaders: 

In year one I didn't realize how important they were. We developed them, but now when we’re 
going into year three, the competencies drive everything we do. Every training we have, you’re 
going to see the competencies slide at the beginning, what competencies are we focusing on, so it 
might be instructional expertise in that training, it might be achievement focus.... Every principal 
and AP has a learning plan, where they sat down with their supervisor and created goals based on 
our competencies. It’s the language that is now being used across the district. 

Hillsborough County formed an Alignment Committee composed of the same individuals, including princi- 
pals and district-level staff, who had developed the original standards and competencies. The committee 
used the eight competencies as the lens for reviewing and modifying preparation programs and support at 
multiple career stages. The idea was to organize curricula for the sequence of two district-run preparation 
programs so that they would not be redundant with each other but would take participants “deeper and 
deeper” into the development of their competencies, as one district staff member put it. The committee also 
worked to align assessments with the competencies. 

Indeed, the focus on standards and competencies reportedly helped Hillsborough County conceptualize 
many pipeline components, from preparation through evaluation, as a continuum in which the district 
could collect standards-based data on individuals and use the data for selection as well as support. With 
each aspiring principal’s experiences and performance summarized in the Leader Tracking System, the 
aligned programs and assessments were expected to pay off in better choices of new principals. A district 
official explained: 

Now when a candidate tries to get into our pool at the beginning, the Future Leader Academy [for 
aspiring assistant principals], they’re assessed on those eight competencies. That information is in 
our electronic system, our Leader Tracking System. And then as they progress through the pipeline, 
you’re going to continue to collect data about those eight competencies to where when you get to 
the end, you really would be able to say, “This person is one of the strongest we have when it comes 
to this competency,” and so you would be able to make a better match, better hiring decisions. 


Revising Standards and Competencies Based on Experience 

Over the years, all the districts revisited their standards and competencies, adapting and revising them to 
meet the practical challenges encountered in leader preparation, hiring, evaluation, and support. The refine- 
ment of leadership standards over time reflected lessons learned in the development of tools and materials 
for leader development and assessment or the refinement of standards for other role groups. 

For example, in spring 2014 Charlotte-Mecklenburg overhauled its selection procedures for aspiring princi- 
pals (as described in Chapter 5). A district official described the process of working with other district lead- 



ers to figure out how candidates might demonstrate the skills and beliefs described in the district’s leader 
competencies: 

We really worked with them on [questions like,] for belief in students, what does that mean? To what 
level do we expect our people to demonstrate that, to be successful as a first-year principal? And so 
that is when we really began the redesign work and came up with our new competencies, new rubrics. 

Denver leaders, seeking to clarify the district’s key expectations for all educators, worked toward alignment 
among the standards documents they had developed for different positions in the schools and district office. 
The documents they reviewed in this process, called frameworks, included the district’s longstanding 
Framework for Effective Teaching and a new framework for central-office leaders, as well as the School 
Leadership Framework designed for principals and assistant principals. A district official described an ex- 
ample of useful new insights that came from the systematic comparison of the frameworks: 

We wanted them to align across the district. Everybody's working on their own framework, and so we 
pulled a group together and we put all of our frameworks together and asked, where did they align 
and where did they not? And ours was pretty solid, the School Leadership Framework, except every- 
body else had this “personal leadership” and we didn’t. So that’s where I said, “Fla! There's our gap.” 


New York City's Approach to Principal Standards 

New York City took a distinctive approach to the development and application of standards for principals. 
The School Leader Competencies that it put forth in 2012 were something of a placeholder, lightly adapted 
from an existing document that reportedly was not viewed as particularly useful. Meanwhile, though, an 
in-depth process had been launched with the aim of reframing the principal evaluation instrument to align 
closely with the Quality Review, which was a component (alongside student performance) of the district’s 
consequential evaluation system for schools. 


21 


The Quality Review incorporated rubrics and indicators for assessing each school’s instructional core, sys- 
tems for improvement, and school culture. District leaders reasoned that the standards for principal evalua- 
tion should be organized around the qualities that the city expected to see in its schools. They set out, 
therefore, to map backward from the Quality Review Rubric to identify the principal competencies associ- 
ated with each school-level indicator and to use the resulting standards for principal evaluation — and, at 
the same time, to align all Principal Pipeline Initiative components to the same standards. 

Building acceptance for this approach across offices, with the state, and with the administrators’ union 
took time. By 2013, however, the new Principal Performance Review was in place, with indicators aligned 
to the Quality Review Rubric. 


According to interviewees, the transition was a success. A principal supervisor praised the evaluation indi- 
cators that operationalized the principal standards, observing that they eased the work of evaluating and 
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supporting principals because they reflected “great information that you can capture in a school” and were 
“easy to talk about” with principals, who were already deeply familiar with the Quality Review. A district- 
run preparation program had a curriculum reorganized around the standards. A district official, pointing to 
the value of the alignment effort, offered a rapid-fire listing of its benefits that closely matched the way the 
entire pipeline initiative was supposed to work: the use of a single set of standards could clarify expecta- 
tions for preparation programs, hiring, evaluation, and support; it would facilitate accountability and could 
contribute to shared understanding among multiple stakeholders. The comprehensiveness of this statement 
is striking: 

I think that what's really been working is that we have now a single consistent set of standards that 
is used through the entire trajectory of a principal’s development and then work as principal. So it’s 
the same set of standards that we’re using to select people for a principal preparation program, to 
evaluate their readiness to become a principal, and then to evaluate their work as a principal, as 
well as to drive the support that they’re given in the principal role. I think that it’s helpful in terms 
of making sure that hiring managers, central office, principals in the field, coaches, principal prep 
programs are all on the same page in terms of what it is that we expect of principals and what the 
standards are that we are going to hold them accountable for. I think that’s really been great. 

Before we began this work, prep programs had their own sets of competencies and standards, and 
there were maybe different standards for evaluating people who were interested in becoming prin- 
cipals and then other standards for evaluating ones that were principals. It was a hodgepodge of 
different standards, and I think that we’ve managed to create a coherent understanding across a 
number of different stakeholders. 


Standards for Assistant Principals 

Interviewees in five districts reported that standards for assistant principals were the same as those for prin- 
cipals, but that a principal’s level of proficiency or performance was not expected of assistant principals. 
“Those exact same competencies and beliefs are now being used with our APs, because we expect the same 
skills, competencies, and beliefs,” said a Charlotte-Mecklenburg official — but she then qualified the state- 
ment with, “Possibly to a different level for a first-year AP.” Similarly, a Denver interviewee said that the 
stated leader standards were the same for both roles, but added: 

What was new this year is we developed an evidence guide to support the role of assistant princi- 
pals.... So now we have one school leader framework, two evidence guides. It’s the same frame- 
work, so the same standards for both jobs. But the evidence guides are differentiated, so what's 
“effective” at the AP level would look slightly different than what's “effective” at the principal level. 

A Prince George’s County official explained that the standards for assistant principals closely resembled 
those for principals but that the indicators were expressed in terms of “supporting” the principal in carry- 
ing out the functions that officially belonged to the principal. For example, an assistant principal would be 
expected to “support the principal in” areas such as the school budget. 



In New York City, on the other hand, standards had not been built out for assistant principals in the same 
way that they had for principals. An interviewee commented on this fact, implying that it was an area ripe 
for future improvement: 

An interesting thing in our system is that there’s a very rigorous and well-defined set of standards 
and a process for evaluating both teachers and principals, but not so much for APs. The principals 
have to write up some kind of evaluation report at the end of the year and give them a “satisfac- 
tory” or “unsatisfactory.” 


Summary: Changes under the Initiative, and a Look Ahead 

A mandate to develop standards would not necessarily lead to active, continuing use of the standards, but 
in this initiative that is what ensued. Rather than leave their standards and competencies entombed in 
binders on their shelves, these districts’ leaders put them to active use in structuring principal preparation, 
hiring, evaluation, and support — as later chapters discuss in detail. 

Although no district’s leaders said in interviews that their standards needed a thorough overhaul, incremen- 
tal revisions appeared very likely to continue. Future work on evaluation and support — the initiative com- 
ponent that was scheduled for completion in 2015 — appeared particularly likely to spur further revisions in 
standards and competencies because new issues were likely to arise in the practical application of stan- 
dards. Districts could potentially spend more time reviewing standards for assistant principals. New devel- 
opments in the field of school leadership, such as the expected release of new ISLLC standards in early 
2015, could also be expected to prompt further review of the districts’ standards and competencies. On this 
note, a leader in Denver described the belief that any local standards document should be revisited on a 
regular cycle: 

It’s an iterative, dynamic document.... If we’re constantly changing and improving in the field, if 
we assume that, then this document needs to be responsive to the standards that are changing for 
the roles. 
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4. Preservice Leader Preparation 

r he second major component of the Principal 
Pipeline Initiative design was a strong district 
role in shaping the preservice preparation of 
high-potential aspiring leaders. Districts embraced this 
aim in two major ways: by launching or expanding the 
programs that they themselves managed; and by seeking 
greater influence over external programs— those in 
nearby universities or nonprofit organizations-whose 
graduates they hired. 

The initiative’s theory of action featured the latter ap- 
proach, with the expectation that districts’ power as 
“consumers” of program graduates gave them the op- 
portunity to change the local ecosystem of preparation 
programs, including the university programs that led to 
state administrator certification (Orr et ah, 2010). As 
described in this chapter, the initiative provided funding 
and assistance for the use of Quality Measures, a facili- 
tated process in which districts and “preferred 
provider” programs together assembled and reviewed data on each program’s design and results so that 
programs in districts, universities, and other partner organizations could benefit from data-informed im- 
provements. The initiative also encouraged districts to give formal preference in hiring to the graduates of 
the programs that district leaders deemed to be of high quality, thus creating a market-based incentive for 
programs to meet the district’s expectations. 

The evaluation team studied the districts’ activities in leader preparation, analyzing the existing state of 
preservice preparation and the changes that took place under the initiative. This chapter analyzes the ways 
in which each district approached the opportunity to reshape preparation, the relationships forged with 
external partners, the experiences and preparation reported by novice principals, and preparation for as- 
sistant principals. 


Takeaways 

1. Districts developed their own, district-run 
programs as part of the multi-year process 
of formal preparation. 

2. Districts also formed partnerships with 
preparation programs outside the district. 
This included collecting and reviewing data 
on program features and outcomes. 

3. The preparation programs serving these 
districts measured up reasonably well by the 
yardstick of past research on exemplary 
preparation programs, according to 
principal reports. 


Revising the Process of Formal Preparation 

Between 2011 and 2014, districts expanded and deepened the programming for aspiring principals. Each 
district had made choices about the preparation pathways that it would allow before the initiative began, 
and some made changes in requirements under the initiative. The pathways can be described as falling into 
three general types (Exhibit 7). Two states, Georgia and Florida, constrained the options by requiring uni- 
versity preparation for certification or licensure, while the other four states — Colorado, Maryland, New 
York, and North Carolina — had also approved alternate routes to principal certification. 
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Exhibit 7: Principal preparation pathways 
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In the first pathway, “university preparation only,” aspiring leaders paid university tuition (some- 
times with a district subsidy) for a program in which they could participate while working full-time. 
The program qualified them for state certification as an administrator. Our data show that in prac- 
tice most graduates went on to work as assistant principals before being tapped as principals. 


A different pathway was the “alternate route with a residency. ” In selective nonprofit programs 
that worked in collaboration with a district, aspiring leaders received fellowship support to leave 








their jobs and take a residency of a year or more as an apprentice leader, with on-the-job mentor- 
ing and group activities designed to develop school-leadership skills. At the end of the residency 
they were eligible for principalships. Actual placement rates as principals or assistant principals 
varied by program, district, and year. 

In a third pathway, “multi-stage preparation with a district final-stage program, ” aspiring leaders 
first went through the university preparation required for initial administrator certification, then 
served as assistant principals. After two or three years in that position they could be selected for a 
district-run, tuition-free program as a final stage of preparation. In that program, while continuing 
to work as assistant principals in their existing school or “residents” in another school, they partic- 
ipated in classes and practical activities designed to inculcate the district’s philosophy and practice 
of school leadership. During or after that final program, most applied to become principals. 


The third pathway, including participation in a district-run preparation program for principals, gained policy 
support across districts under the initiative (Exhibit 8). Gwinnett and Hillsborough Counties reaffirmed their 
commitment to the third pathway of multi-stage preparation. Both districts were committed to a vision in 


Exhibit 8: 

Pathways as of 2011 and pathway changes made by 2014, by district 

District 

2011 

2014 
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Charlotte-Mecklenburg 

University preparation or alternate 
route was required. 

University partner programs had been added. 
Alternate route was being phased out. 

Denver 

University preparation or alternate 
route was required. 

University partner programs had been added. 

A district final-stage program had been added but 
not required. 

Gwinnett County 

Multi-stage. A district final-stage 
program was required. 

University partner programs and a district 
program for aspiring APs had been added. 

No change was made in requirements. 

Hillsborough County 

Multi-stage. A district final-stage 
program was usually required. 

University partner programs and a district 
program for aspiring APs had been added. 

The district final-stage program was required in 
all cases. 

New York City 

All three pathways were available. 

University, programs, alternate-route partner 
programs, and district programs had been added. 

No change was made in the types of pathways 
available. 

Prince George's County 

University preparation or alternate 
route was required. 

University partner programs had been added. 

A district final-stage program had been added but 
not required. 
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which the district itself took the lead in preparing its future principals, and both used some of their grant 
funds to enhance their existing final-stage programs in alignment with their principal standards. Gwinnett 
County lengthened and redesigned the clinical component of its final-stage Aspiring Principals Program. 
Hillsborough County overhauled its final-stage Preparing New Principals program for experienced assistant 
principals, increasing the number of on-the-job activities designed to build participants’ skills and adding a 
mentoring component, and it strengthened the policy of selecting new principals from among graduates of 
this program. Both districts also forged partnerships with nearby university programs. 

Denver and Prince George’s County incorporated this third pathway into principal preparation. Each added 
a new, district-run program for sitting assistant principals whom the district identified as especially promis- 
ing candidates for the principalship. While the new programs were not mandatory prerequisites for princi- 
pal positions, the districts promoted them as a desirable step. Denver’s Learn to Lead program, supported 
by the grant, provided a one-year residency at either the current school or a new school in which partici- 
pants took on additional principal-like roles and developed their leadership skills and perspectives. Prince 
George’s County’s Aspiring Leaders Program for Student Success, developed in partnership with the 
National Institute for School Leadership under the grant, provided a structured curriculum and trained 
mentors. Neither district took other preparation options off the table, however, and both broadened their 
partnerships with universities. 

Charlotte-Mecklenburg and New York City broadened their portfolio of options. They continued to in- 
clude university preparation and alternate routes as ways of qualifying to become a principal, rather than 
requiring or strongly encouraging an additional program. They also moved to strengthen district influence 
over recruitment and selection of candidates for preparation through all programs. Specifically, Charlotte- 
Mecklenburg supported new, specialized program options in two nearby universities in addition to the pro- 
gram that it had previously developed with a third university. New York City, using something of an 
“all-of-the-above” strategy, supported new or redesigned options in universities, expanded the options that 
the district itself offered at each stage of preparation, and worked with its nonprofit partners who were 
adding program options for aspiring leaders. 

A Strong District Role in a Lengthy Preparation Process 

As the districts worked to influence principal preparation, they were mindful of the many years that could 
elapse between starting preparation and becoming a principal. They had agreed to participate in an initia- 
tive that called for preparing substantial numbers of new principals in new ways by early 2015. It is not 
surprising that all six districts started by reshaping the last step in formal preparation, the one closest to ap- 
pointment as a principal. As described below, five of them quickly developed or sought to strengthen in- 
house programs designed to prepare selected assistant principals for success as principals. It would take 
longer to have an effect on incoming principals by way of improving the university-based programs leading 
to initial certification. 

Our survey data show the length of time that preparation could take. Across districts, survey respondents in 
2014 (those who had started on the job between March 2011 and Lebruary 2014) reported a median of 78 
months — six and a half years — from the beginning of their first leadership preparation program to their first 



day on the job as a principal (Exhibit 9). The median was 60 months, or five years, for survey respondents in 
the previous year (a partially overlapping group who had started on the job between spring 2010 and spring 
2013). The median length of time varied across districts. In 2014 it ranged from about three years in New 
York City to nine years in Hillsborough County. This meant that if Hillsborough County had been able to 
influence changes in all the university programs that prepared its future principals, starting immediately after 
grant award in 2011, the effects could potentially be seen in around half of new principals by 2020. 



Exhibit reads: The median number of months between starting a leadership preparation program and starting a job as a principal was 55 months for the 23 
first-, second-, and third-year principals responding in 2013 and 72 months forthe 28 responding in 2014 in Charlotte-Mecklenburg. 

Source: Principal Survey for “Evaluation of the Principal Pipeline Initiative,” 2013 and 2014. 

For five districts, then, a key priority at the start of the initiative was a program for high-potential assistant 
principals that the district itself operated. Charlotte-Mecklenburg, the sixth district, relied instead on ex- 
panding its formal agreements with local universities, supplementing its contractual partnership with the 
New Leaders alternate certification program. Having launched partner relationships before the initiative 
began, Charlotte-Mecklenburg was already seeing principal applicants prepared in the partner programs 
and could hope that the programs would supply a good proportion of new principals soon. 
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District-Run Programs 

District leaders saw their own preparation programs as having some advantages over external programs: 

A selective admissions process gave the district a checkpoint for spotting high-potential leaders. 

District leaders who conducted class sessions had a chance to observe and size up the rising cohort 
of principal candidates. 

Program content could be tailored to the principalship routines and priorities of the district, im- 
mersing the aspiring leaders in those routines and priorities. 

Seeing prospective principals repeatedly over the course of a district-run preparation program gave the dis- 
trict a strong head start in principal selection, according to interviewees. One Gwinnett County official suc- 
cinctly observed of the district’s preparation program: “It’s a full-year interview.” In districts with selective 
internal preparation programs, district hiring managers and others could get to know future candidates and 
see them engage in authentic leader responsibilities tied to leadership standards. The internal programs also 
provided an opportunity to counsel people out of the path to principalship or give them an opportunity to 
withdraw after gaining a better understanding of the job. Another Gwinnett County leader said of partici- 
pants in that district’s sequence of two preparation programs: “We get two shots at screening them before 
they get to be principals.” 

The districts could use their leader standards both in setting admissions criteria and in developing or refin- 
ing program curricula. Hillsborough County’s work on alignment, for example, included use of the dis- 
trict’s eight key competencies in the criteria for admission to district-run preparation, and the alignment 
committee was hard at work in 2013-14 ensuring that at each stage of preparation the aspiring principals 
would have opportunities to develop those specific competencies. In New York City, the director of one of 
the district-run programs described recent changes made to align with the Quality Review Rubric, which is 
now the key standards document (as described in Chapter 3): 

The general scope and sequence of the program has been adjusted to fit the department’s priorities. 
This year, it was decided that we should align our content more closely to the Quality Review. . . . 
Our faculty also participates in the PPO [Principal Practice Observation] visits, which are the prin- 
cipal rating visits. Our faculty know exactly what people are being rated on.... Literally, one of my 
people can walk out of a PPO, ... go right to class, and say, “This is what you're being required to 
do as a principal right now. This is what I'm seeing, and this is how you could do better.” 


Partnerships with Programs Outside the District 

Expanding and deepening partnerships with principal-preparation programs outside the district was a key 
aim of the Principal Pipeline Initiative (Turnbull, Riley, & MacFarlane, 2013). Two districts, Denver and 
Charlotte-Mecklenburg, had formal partnerships with universities that had begun well before the Principal 



Pipeline Initiative. Each program was tailored closely to district requirements. Denver Public Schools and 
the University of Denver created the Ritchie Program for School Leaders together in 2003, and the program 
was continuously co-designed and co-led by the district and university. Recruitment and admissions were 
carried out collaboratively; applicants’ data were shared with the district; the program’s project-based cur- 
riculum and the assessment of participants were aligned with the district’s leader standards — for which one 
of the original sources had been the Ritchie curriculum. 

Charlotte-Mecklenburg worked with Winthrop University to develop the two-year Leaders for Tomorrow 
program, which admitted its first class in 2008. A top district leader participated directly in selecting the 
students to be admitted each year. Interviewees in both the district and the university described the district 
as the “customer” for Leaders for Tomorrow. The district required that the program be aligned with the 
district’s strategic plan, and the program director fine-tuned the curriculum and activities to align with new 
district priorities. 

Denver and Charlotte-Mecklenburg used their existing university relationships as models for new start-up 
programs in universities: 

Denver expanded its partnership with the University of Denver to include an additional program 
for emerging leaders. And a new partner program at the University of Colorado-Denver was de- 
signed to focus on leadership that would effectively serve English learners. 

Charlotte-Mecklenburg launched the School Executive Leadership Academy in partnership with 
two schools within Queens University, the business school and the education school. It also initi- 
ated a program at the University of North Carolina-Charlotte, with three sub-programs of varying 
duration including a five-week summer intensive program designed to prepare principals for high 
schools. 

New York City and Prince George’s County worked with nearby universities to revamp or initiate prepara- 
tion programs, using grant funds as part of the support for these efforts: 

As of 2013-14, new or revised programs for principal certification were operating in partnership 
with New York City at Bank Street College, Teachers College of Columbia University, Lordham 
University, and Relay Graduate School of Education. 

Bowie State University was partnering with Prince George’s County in a program for principal 
preparation that had become more closely aligned with the district’s priorities in school leadership. 
The district also initiated discussions with several other institutions, including the one that had the 
largest number of alumni working as school leaders in the district, McDaniel College. 

Gwinnett County and Hillsborough County formalized their relationships with existing university programs 
for administrator certification. District officials met regularly with the program leaders in selected universi- 
ties. They provided input that resulted in at least some curricular or programmatic change in university pro- 
grams, and both districts provided the services of district staff as instructors or coaches in those programs. 
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Most relationships with nonprofit preparation programs also continued or grew through spring 2014. New 
Leaders continued to work in partnership with Charlotte-Mecklenburg, New York City, and Prince George’s 
County for principal preparation. Its early-stage program for potential future principals, Emerging Leaders, 
was also operating in New York City. The New York City Leadership Academy continued to prepare princi- 
pals for New York City and, under the Principal Pipeline Initiative, had begun a new, two-year program to 
prepare aspiring assistant principals who could join forces with principal candidates and apply as a jointly 
prepared leadership team for a school. The teams then received intensive coaching under other funding. 

One program partnership was set for termination, however. As of spring 2014, Charlotte-Mecklenburg had 
chosen to end its contractual partnership with New Leaders for principal preparation for cost reasons. This 
was the only partnership nearing termination at this point in the initiative. 

All districts were selective in choosing partners, assessing the likely benefit from the working relationship. 
The programs that districts partnered with met criteria such as: philosophical alignment with the district’s 
beliefs about leadership competencies and how to develop them; willingness to engage in frequent commu- 
nication; and openness to making program changes. 


Quality Measures and the Use of Data 

A vehicle for strengthening partnerships under the Principal Pipeline Initiative was use of the Quality 
Measures for Education Leadership Systems and Programs. Designed by Education Development Center 
with Wallace support, Quality Measures is a suite of tools that include indicators and rubrics for assessing 
leader preparation programs. It is organized around six program components that are associated with pro- 
gram effectiveness in leader preparation, drawing particularly on the work of Darling-Hammond and col- 
leagues (2007). The components are: course content and pedagogy; clinical practice; recruitment and 
selection; and three components related to graduate performance outcomes (knowledge, skills, and compe- 
tencies; responsiveness to market demand; and impact on school, teacher, student performance). 

The Principal Pipeline Initiative provided facilitators, training, and protocols for the Quality Measures 
work, which was intended to help districts and program leaders assess these program components in a 
data-based process. Leaders of programs in districts and partner organizations gathered evidence on the ex- 
tent to which their programs met criteria in each of the components listed. At each site, program represen- 
tatives then met together with district staff and Quality Measures facilitators to review each program’s 
evidence and rate it on a rubric of indicators related to each component. 

A total of 32 preparation programs operated by or partnering with the six districts had at least begun the 
process of Quality Measures as of August 2014. They included: 

Twenty-two university-based preservice programs, including five working with Gwinnett County, 
five with Hillsborough County, four with Charlotte-Mecklenburg, four with Prince George’s 
County, three with Denver, and one with New York City. 



Six district-run programs, including two from New York City and one each from Denver, Gwinnett 
County, Hillsborough County, and Prince George’s County. 

Two of the New Leaders alternate-route programs: one from New York City and one from Prince 
George’s County. 

Two alternate-route programs developed and led by other organizations: one that worked with 
Denver and one that worked with New York City. 

Across all programs that had completed the full review process, Education Development Center reported 
that the evidence related to course content and pedagogy was most often rated “well developed” or “devel- 
oped.” The evidence related to graduate program outcomes was much weaker, generally falling in one of 
the two lowest rating categories. The review teams recommended that districts and providers develop or 
improve tracking systems to determine graduates’ employment status, to support principals during induc- 
tion, and to “collect relevant longitudinal data” (Education Development Center, 2014, p. 11). 

In assessing the effects of the Quality Measures process on both programs and partnerships, our intervie- 
wees pointed to the value of systematically reviewing and discussing program features and exchanging in- 
formation. A typical positive comment was the following, from a leader of a university-based preservice 
program who compared the process favorably to traditional accreditation: 

We happen to think our program is pretty good, but we learned so much in the three-day process with 
other teams. We immediately started making refinements [to our program]. It was a really informative 
process. The protocols are so helpful in way that is different from the accreditation reviews. 

Interviewees also commented that the collaborative work on the Quality Measures process strengthened 
working relationships between districts and external programs. Together with the recommendations for 
tracking systems, this built receptivity among partner programs for continuing feedback about their opera- 
tions and their graduates’ performance. 

The districts’ Leader Tracking Systems, when populated with data spanning the entire career trajectories of 
leaders and potential leaders, will expand the possibilities for such feedback. One of many anticipated ca- 
pabilities of the systems is the ability to tie an individual’s principal preparation experiences to data such as 
evaluation ratings and school performance. Over time, district leaders recognized, analysis and communica- 
tion of such findings could inform program improvement and contribute to program accountability. This 
use of the system had not begun as of 2014, however. 
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Sources of Districts' Power in Relation to Preparation Programs 

A particularly influential study in the design of the Principal Pipeline Initiative was that of Orr and col- 
leagues (2010), who classified districts’ relationships with preservice preparation as reflecting the stance of 
a competitor, customer, or collaborator. Under the initiative, we found that all of the districts wielded the 
power of “customers” while also “collaborating” with partner institutions and organizations. All districts 
behaved as customers with partners and potential partners, for example by establishing leader standards 
that the district would look for in program graduates. Each district also followed the initiative design by 
identifying a set of preferred providers, or programs that it believed were most aligned with district inter- 
ests, and collaborating with them at least to some degree. The new and stronger relationships with external 
programs show that all six districts were willing to act as “discerning customers” and as “collaborators,” 
in Orr’s terms. 
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A “competitor” stance could be seen, however, in five districts’ support of their own, district-run preparation 
programs. Some district-run preparation options were in fact direct competitors to external programs whose 
graduates qualified for certification. In 2013 Denver won state approval to grant administrator certification 
through its own alternative program. The district used that authority sparingly over the first year, to accom- 
modate individuals whose preparation had not fit the usual mold, but district leaders were considering op- 
tions for future program design. Participants in one of New York City’s district-run programs earned 
certification through an arrangement with Baruch College, but as a practical matter the program was run by 
the district more than by the college. It appeared possible that these districts and others would seek to en- 
large their footprint as producers, rather than consumers, of aspiring leaders who held state certification. 

The initiative design suggested that districts should exercise consumer power by giving formal preference in 
hiring to graduates of particular programs. This idea drew a mixed reception in the districts. Gwinnett and 
Hillsborough Counties did give strong preference to graduates of their district-run programs. Denver and 
Prince George’s County arranged for graduates of their new district-run programs to enjoy some advan- 
tages in the application process of applying to principal positions. Arguably, though, a preference for grad- 
uates of a district’s own program reflected its choice to be a producer of principals, more than a choice to 
wield power as a consumer. 


For every district to some degree, the notion of giving preference to some external programs posed some 
concerns. According to one district leader, whose sentiments were also expressed by others, the lack of for- 
mal preference for partner programs in fact enhanced the district’s power as a consumer that could effec- 
tively pressure these partners (here described as “pipeline programs”) to improve: 


We give no priority to [graduates of] our pipeline programs. They have to clear the same bar every- 
body else does. I don't know how we would justify it not being that way. I think it puts a good 
pressure on our pipeline programs to always be pushing themselves in terms of alignment to the 
district and preparing their candidates for our expectations.... So I think it's a good thing for the 
pipeline programs to know that they're not assured a free pass. That keeps you on your game and 
keeps you tightly connected to the district. . . . They all get feedback from the district in terms of 
where there are gaps in their candidates. 



Principals' Experience with Preparation 


So far, this chapter has largely focused on the institutional relationships between districts and preparation pro- 
grams. We now shift the perspective to that of individual participants. Our data offer two kinds of windows 
into the preparation experiences of the individuals who became principals in these districts. First, we use the 
survey findings to take a retrospective look at novice principals’ description and assessment of their prepara- 
tion. Second, we describe how the districts gave preparation opportunities to their aspiring principals. 


Principals' Perceptions of Their Preparation Experiences 

Survey data give a picture of the preparation of four entering cohorts of principals — preparation that 
largely took place before the Principal Pipeline Initiative. Our questions asked for reflection on “the total- 
ity” of the respondent’s preservice preparation, not specific programs or types of programs. Survey respon- 
dents were reflecting in part on the programs in which they participated in the last year or two before 
taking the principal job, but also on programs in which they participated many years earlier (recalling that 
the median time span between the start of preparation and the start of a principalship was 60 months 
across all districts among the 2013 respondents, 78 months among the 2014 respondents). Our principal 
survey respondents of 2013 became principals in 2010-11, 2011-12, and 2012-13; the survey respondents 
of spring 2014 became principals in 2011-12, 2012-13, and 2013-14. The cohorts who became principals 
in the two earliest years had completed their preservice preparation when the grant was awarded. The co- 
horts who became principals in the next two school years might have had the last year of their preparation 
after the grant award. However, a typical starting time for all respondents’ preparation would likely have 
been in the 2000s. 

Survey responses indicated that preparation experiences predating and immediately following the grant 
award already showed many of the preparation-program features that the initiative called for (Exhibit 10). 
These features included a focus on instructional leadership, field-based activities, and a cohort group of 
peers. These data suggest that the districts were already hiring principals who had benefited from high- 
quality preparation programs as defined in prior research (Darling-Hammond et ah, 2007), consistent with 
the foundation’s intention to select grantee districts that were already carrying out some components of the 
initiative. No statistically significant differences were found in responses from 2013 to 2014. This is not 
surprising, since the groups of respondents overlap substantially. 
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Exhibit 10: 

Principal perceptions of their preparation experiences, 2013 and 2014 


To what extent were the following true of your 
formal leadership preparation training? 

Percent of principals saying to a 
considerable or great extent 

2013 2014 

(N=326) (N=472) 

a. Program content emphasized leadership for school improvement 

82% 

87% 

b. Program content emphasized instructional leadership 

81 

88 

c. My interactions with peers have had a positive influence 
on my personal growth 

80 

85 

d. Program gave me a strong orientation to the principalship as a career 

77 

80 

e. 1 had coursework that included analysis and discussion of case studies 

76 

82 

f. 1 had coursework that included analysis and discussion of field-based 
problems/problem-based learning approaches 

74 

81 

g. 1 had coursework that included a portfolio demonstrating learning 
and accomplishments 

72 

73 

h. 1 have developed close personal relationships with other 
leadership students 

69 

78 

i. 1 had coursework that included field-based projects in which 
1 applied ideas in the field 

67 

74 

j. 1 had coursework that included completion of a capstone or 
culminating project 

64 

73 

k. I had coursework that included action research or inquiry projects 

63 

71 

1. Program content was tailored to district context 

60 

64 


Exhibit reads: In 2013, 82 percent of first-, second-, and third-year principals in the six districts reported that their preparation program emphasized leadership for school 
improvement to a considerable or great extent. In 2014, 87 percent of first-, second-, and third-year principals in the six districts reported that their preparation program 
emphasized leadership for school improvement to a considerable or great extent. 

Source: Principal Survey for “Evaluation of the Principal Pipeline Initiative,” 2013 and 2014. 


Principals gave generally positive assessments of the results of their preparation, from the vantage point of 
their subsequent experience as novice principals (Exhibit 11). Among the 2014 respondents, for example, 
87 percent said that their programs had prepared them well or very well to engage in continuous learning; 
83 percent said the programs had prepared them to create a collaborative learning organization. They also 
reported that they had been prepared well or very well to evaluate teachers and provide feedback (82 per- 
cent), engage staff in decision making (82 percent), lead school change (82 percent), plan comprehensively 
for school improvement (80 percent), and use data (79 percent). Responses in the previous year were statis- 
tically similar. It should be noted, however, that all self-reports in response to these questions presumably 
reflect some upward bias. 





Exhibit 11: 

Principal perceptions of their preparedness, 2013 and 2014 



Percent of principals saying 
well or very well 

How effectively did your leadership preparation training 
prepare you to do each of the following? 

2013 

(N=326) 

2014 

(N=476) 

a. Engage in self-improvement and continuous learning 

88% 

87% 

b. Create a collaborative learning organization 

82 

83 

c. Use data to monitor school progress, identify problems and propose solutions 

78 

79 

d. Engage staff in a decision making process about school curriculum and policies 

77 

82 

e. Evaluate teachers and provide instructional feedback to support their improvement 

76 

82 

f. Lead a well-informed, planned change process for a school 

76 

82 

g. Engage in comprehensive planning for school improvement 

76 

80 

h. Create a coherent educational program across the school 

72 

76 

i. Redesign school organizations to enhance productive teaching and learning 

72 

72 

j. Design professional development that builds teachers' knowledge and skills 

65 

70 

k. Understand how different students learn and how to teach them successfully 

59 

61 

1. Evaluate curriculum materials for their usefulness in supporting learning 

50 

55 


Exhibit reads: In 2013, 88 percent of first, second, and third-year principals in the six districts reported that their preparation programs prepared them well or very well to engage in 
self-improvement and continuous learning. In 2014, 87 percent of first, second, and third-year principals in the six districts reported that their preparation programs prepared them 
well or very well to engage in self-improvement and continuous learning. 

Source: Principal Survey for “Evaluation of the Principal Pipeline Initiative,” 2013 and 2014. 


Research on preparation programs — especially the work of Linda Darling-Hammond and colleagues 
(2007) — helped dictate the program features that the Principal Pipeline Initiative called for. Our principal 
survey allows an exploratory comparison of principals’ reports on their preparation with those of the 
Darling-Hammond team’s two groups of respondents: graduates of the programs highlighted as exemplars, 
and a national comparison group of principals. 

These districts appeared to enjoy the benefit of relatively strong preparation programs. The strength of 
preparation at the start of this initiative and shortly after is suggested by the similarity between the survey 
responses of novice principals in these districts and those of graduates of the exemplary programs studied 
by Darling-Hammond and colleagues. Further, the survey responses found in this study were more favor- 
able in some respects than the responses among that study’s national sample. This held true for these gradu- 
ates’ assessments of program content and instructional approaches and their self-reported preparation for 
specific aspects of leadership. The detailed comparative findings appear in Appendix B. 
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In interpreting these findings, it is important to consider that the program features identified as unusual 
strengths in the middle of the past decade may have become more widespread everywhere, not just among 
the programs preparing principals for these districts. Because these survey instruments have not been used 
to gather nationally representative data on leader preparation in recent years, we cannot know how the 
programs serving these districts would compare with others nationally. 


How Districts Worked with Aspiring Principals 

Selection into a district program of principal preparation often meant that the individual selected had a good 
chance of becoming a principal. Interviewees in three districts described their programs in similar terms: 

Hillsborough is different from many other districts in the sense that selection into even [assistant- 
principal preparation] and certainly [principal preparation] is a pretty straight line to a hiring 
process. 

We’ve found quickly that [those entering Denver’s program] needed to be, at best, six months away 
[from being principal-ready], because we start interviewing and selecting for principal jobs virtually 
in January.... So we’ve changed our language in that it’s not a principal preparation program, but 
more of a “transition into the principalship” type of program. 

You are clearly told that when you turn in application for [Gwinnett’s program], you’re signaling 
that you’re ready to be a principal the next day. 

These programs had rigorous selection processes that used performance tasks and interviews in selecting 
candidates, with ratings by groups of trained staff using instruments aligned with the leader standards. 
Prince George’s County also used rigorous selection, requiring a video demonstrating on-the-job leadership, 
along with recommendations and a writing sample. The difference in Prince George’s County was the con- 
text of limited available positions: very few new principals were being appointed, and graduates of the new 
preparation program faced stiff competition for those positions. 

Another example of a close connection with the hiring process was in one district-run program in New 
York City, where participants were nominated by leaders of the sub-district units called clusters. Each clus- 
ter nominated approximately 20 individuals seen as highly promising candidates to assume principalships 
within several years. An interviewee who worked with the program said: 

Given that our goal is to provide a pipeline of leadership for each cluster, if a cluster’s going to en- 
dorse someone, that means a lot. 

The districts’ own preparation programs were used as a way to get to know each participant’s strengths 
and potential. Ongoing participant assessments aligned with leader standards — standards that also shaped 
the principal job descriptions and hiring criteria — contributed to the sense that these programs offered a 
strong position for future hiring. 



Another way in which all six districts worked with aspiring leaders was to provide them with structured in- 
formation about preparation opportunities and planning their career paths. In New York City, the diversity 
of school opportunities and preparation programs presented a large-scale challenge in helping each aspiring 
leader find the right pathway. This was described as an issue in 2012. The district then developed a page on 
its website with capsule descriptions of all the partner preparation programs and links to their websites, 
and that page became the second-most widely viewed page on the district’s site. But providing enough in- 
formation to aspiring leaders was still seen as an issue in 2014, when a district official said: 

I think it’s actually sort of confusing to people in the field because it's hard for them to differentiate 
which program is really right for me. I think that it would be smart for us to really try and get 
clearer about which programs are best for who.... I don’t think we have been as clear as we can be 
about which programs are best suited for different career aspirations. 

Pointing to the wide range of aspiring principals’ situations and purposes, this official elaborated on the 
challenge of helping to clarify: 


...which programs are really best if you’re a teacher who needs her certification and wants to be- 
come an AP versus which program is really designed for a more seasoned educator who’s serious 
about becoming a principal right away, and even within that, which program is best for somebody 
who wants to go in and turn around a struggling school versus somebody who sees themself as tak- 
ing over the school that they've been working in for ten years. 
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Preparation for Assistant Principals 

Districts varied in the extent to which they had crafted programs for assistant principals, either to prepare 
aspiring assistant principals or to support novice assistant principals on the job. (The latter type of program 
is discussed in Chapter 6 of this report.) In most districts, university programs leading to initial administra- 
tor certification constituted the formal preparation that assistant principals received. However, the two dis- 
tricts with the most tightly prescribed pathways to the principalship launched similar pathways to the 
assistant principalship early in the initiative. Gwinnett County instituted a more rigorous selection process 
for the Aspiring Leader Program that it offered to would-be assistant principals. Hillsborough County 
started up a selective Future Leaders Academy for individuals who had their initial administrative certifica- 
tion (based on their university preparation) and hoped to become assistant principals. Each of these dis- 
tricts also provided structured mentoring for its new assistant principals, and Hillsborough County was 
considering adding more mentors in order to shrink their caseloads. Charlotte-Mecklenburg also had an in- 
duction program for novice assistant principals. 
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Summary: Changes under the Initiative , and a Look Ahead 


For district decision makers, although a consumer approach offered potential improvement in their princi- 
pal corps over the long term, the direct approach of preparing their own new principals also held appeal. 
Thus, the launch or expansion of district-run preparation programs was a feature of early implementation 
in five districts (all but Charlotte-Mecklenburg). 

With preparation in universities, alternate-route programs, and district offerings, the principals newly hired 
in the years from 2011 through early 2014 reported that they were well prepared for their responsibilities 
in such key areas as instructional leadership. Their responses also suggested that cohort models, problem- 
based learning, and clinical experience were widespread across preparation programs that served these dis- 
tricts before the grant award, although tailoring of content to the district was less so. Overall, their survey 
responses did not differ greatly from those of graduates of exemplar programs surveyed by Darling- 
Hammond and colleagues (2007). 

Districts were creating ways to follow the progress of principal aspirants and novice principals through 
preparation and beginning practice, gathering data from their own and partner programs and also working 
more intensively with aspirants. Internal gatekeeper programs gave district and school leaders first-hand 
knowledge of individuals who might soon be in the hiring pool. As the districts worked with partners on 
preparation and novice support, they could track individuals’ progress, and new data-tracking systems will 
improve their ability to do so. This information can potentially continue to inform redesign and fine-tuning 
of leader preparation in these sites — but also is of use in a process that is even more salient to district lead- 
ers, that of hiring school leaders. We discuss that process next. 



5. Selective Hiring and Placement 


D istrict leaders saw tremendous potential benefit 
in reshaping the procedures and criteria that 
they used in hiring principals. Under the grant, 
they sought to lay groundwork for the actual hiring of 
principals through earlier selection stages that winnowed 
aspiring school leaders, and through the compilation of 
data on these aspirants’ experience and performance. 

The districts also worked to use available data in succes- 
sion planning as well as principal selection. 


District leaders expressed hope that a systematic set of 
steps for aspiring principals’ advancement could ensure 
stronger rosters of candidates available for hiring as 
principals. Under the grant, every district introduced 
one or more new occasions for a standards-based as- 
sessment of aspiring principals. In New York, new as- 
sessments provided information; in the other districts, 
they not only provided information but also opened or 
closed doors to advancement. 


Takeaways 

1 . Districts saw value in adding new stages 
of assessment and selection for aspiring 
leaders, including the use of talent pools. 

2. Standards-based rubrics, interview 
protocols, and performance tasks offered 
more systematic ways to assess candidate 
skills. 

3. Succession planning and placement 
discussions engaged top-level district 
leaders. 

4. Leader Tracking Systems showed promise 
of organizing the data on experience and 
performance into accessible candidate 
profiles for use in placement decisions. 
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Multi-Stage Selection and Talent Pools 

For these districts, restructuring the avenues to positions in school leadership was not solely a matter of im- 
proving preservice preparation. The restructuring included changes that were closely related to hiring: 
building talent pools of aspiring leaders, rigorously vetted, whom the district considered ready to advance 
in the hiring process. Before grant award, four of the districts (Denver, Hillsborough County, New York 
City, and Prince George’s County) essentially accepted applications for open principal positions from any- 
one holding the state-required certification. There were some barriers to entry in these districts — 
Hillsborough strongly preferred candidates from within the district, especially graduates of its Preparing 
New Principals program; New York City had a “hiring pool” of license holders who had met a few addi- 
tional requirements set by the state — but in general the procedure in these four districts was to post open 
positions and consider the applications that came in. 

Charlotte-Mecklenburg and Gwinnett County had a different approach before the grants were awarded: 
each had instituted a preliminary, district-run selection stage for aspiring leaders who wanted to be consid- 
ered for specific positions. Gwinnett required that aspiring principals participate in its selective Quality-Plus 
Leader Academy before winning principal jobs. Charlotte-Mecklenburg had created “talent pools” for 
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would-be principals and APs, with selection procedures for entry, and expected to fill all open leadership 
positions from among the candidates who had cleared the bar for entry into these pools. 

By 2014, all districts had selection stages and processes of assessment that were prerequisites for a princi- 
palship application from inside or outside the district. The other four districts joined Gwinnett and 
Charlotte-Mecklenburg in having pre-application selection processes: 

In Hillsborough, completion of Preparing New Principals quickly became a more tightly enforced 
requirement for principal candidates, and for future cohorts, entry into that program would also 
have to be preceded by completion of the new Future Leaders Academy, which also had a selective 
admissions process. 

Denver and Prince George’s County instituted selective pools for aspiring principals. 

New York State tightened its entry requirements for the hiring pool, and New York City intro- 
duced a new set of assessments for candidates who qualified for that pool. While no candidate was 
disqualified on the basis of performance on these district assessments, the scores became part of the 
individual records provided to hiring managers. 

Gwinnett County also made changes. Like Hillsborough County, it added a selective program to prepare 
assistant principals, and the data gathered at that stage became part of each aspiring leader’s dossier. 

Selection into a talent pool as a prerequisite for consideration in hiring promised to streamline the filling of 
principal vacancies by limiting the number of applications to be considered at the time that each vacancy 
arose. This was a priority, as seen in the comments of district staff who either complained about a deluge of 
applications for each open position or expressed relief that the tide had been stemmed. District managers in 
Prince George’s County cited their vivid memories of the numbers: “We put out a principal posting and 
over eight hundred applied,” said one. 

A Denver principal supervisor summed up the benefits of instituting a talent pool in this way in 2014: 

Looking at the principal pool, we have had several changes and have finally gotten to giving us bet- 
ter options for principal candidates. The screening process for candidates to enter into a pool has 
greatly increased the level of expertise and quality of principals. Doing all the pre-work up front 
has led to higher-level candidates. 


Issues in Pool Selection 

Although the potential advantages of talent pools were evident, filling a pool with the right number and 
types of candidates has not been simple. Charlotte-Mecklenburg’s experience is instructive. Having set up 
talent pools in 2011, the district repeatedly modified the procedures and criteria used for selection as criti- 
cisms arose. One key issue was the sheer size of the pool: the pool was seen at some times and from some 



perspectives as too small, but at other times and from other perspectives as too large and unselective. As of 
2012, for example, district leaders felt the pool was short on appropriate candidates who could lead high 
schools or establish good community relationships in one part or another of this demographically diverse 
district. Principals asked why the candidates they recommended were not accepted into the pool. In re- 
sponse to these concerns, the process was revised and criteria relaxed in order both to yield a larger pool 
and to accommodate principals’ recommendations. In the following year, though, selectivity again ramped 
up because the pool was seen to be producing marginally qualified candidates for vacancies. In short, 
changes were made because the consequences of particular design choices emerged over time. One intervie- 
wee emphasized the risk of discouraging aspiring leaders when either too few or too many were entering 
the pool: 

We’ve seen it ... go from a very high bar to a concern that good people were not getting into the 
pool. ... So what’s that sweet spot in terms of the rigor of your selection process that isn’t discour- 
aging to those high-potential people that just still aren’t clearing the bar to get into the principal 
pool? ...We open it up ... and we have a whole bunch of people sitting in a pool, some or many 
not very qualified, and they get discouraged because they never get interviews and they never get 
selected and they sit there.... So I don't know that we've found that sweet spot. 

Charlotte-Mecklenburg’s experience over the years suggests that the districts that more recently introduced tal- 
ent pools may also want to make design adjustments in the future. For example, as of 2014, Prince George’s 
County continued to receive large numbers of applications from individuals in its pool. Although two intervie- 
wees commented that the abundant supply of qualified candidates was “a good problem to have,” others can- 
didly acknowledged that the need to review hundreds of applications could be evidence of flaws in the system. 
In no district did leaders say that they had found the right level of selectivity for their pool. 


More Ways to Assess Candidates' Skills 

In 2011, Gwinnett County stood out among the grantees for the relatively elaborate performance-assess- 
ment procedures it used in assessing aspiring leaders. With simulations that included several role-play inter- 
actions, in addition to the chance to observe potential principals during their participation in the Aspiring 
Principal Program of the district’s Quality-Plus Leader Academy, the district approached its hiring decisions 
comfortable in the knowledge that it had assembled a great deal of relevant data on each candidate. 

Under the grant, all the other districts incorporated some form of performance assessment at one or more 
selection stages, and Gwinnett County continued to review and refine its procedures as well. Prince 
George’s County moved quickly to revamp its hiring procedures, seeking a basis for hiring that would differ 
from the old-fashioned system widely described as “who you know.” “We wanted to eliminate the old-boy 
network,” said a district official in spring 2012. Recommendations and “popularity” were outweighed by 
new measures that offered promise of greater objectivity and precision. Prince George’s County used its ini- 
tial leader standards to reshape its candidate assessments in 2011-12. A district official explained in 2012: 
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People must demonstrate that they have the ability to carry out standards — they conduct an obser- 
vation, they analyze data, they create a vision statement. All these things are a part of the work 
that has emerged from the standards. 

The other four districts (Charlotte-Mecklenburg, Denver, Hillsborough County, and New York City) devel- 
oped new assessments for aspiring leaders at least two years after grant award, a process that helped shape 
the refinement and elaboration of leader standards. As described in Chapter 3, the standards themselves set 
an overarching framework within which each district built out more detailed statements of principal com- 
petencies and, increasingly, specified the evidence that would reveal each competency. Here we discuss the 
assessments used at any stage of the hiring process, including selection into a talent pool as well as any later 
stage in which candidates participated in a further assessment for specific positions. 

The increased use of practical demonstrations in the hiring process was apparent in survey respondents’ re- 
ports. Among the two cohorts of principals who started on the job in 2010-11 and 2011-12, practical 
demonstrations had been part of the application experience (in applying for a talent pool or for a principal 
position) for 31 percent, across districts. The rate of practical demonstrations rose to 50 percent for the 
two later cohorts, those who started on the job in 2012-13 and 2013-14. This was the selection procedure 
that showed the greatest increase over time. 4 

In overhauling their assessment procedures for aspiring leaders, then, these districts looked for practical 
ways to gather evidence aligned with their leader standards. Three of them (Charlotte-Mecklenburg, 

Denver, and Hillsborough County) enlisted consulting help in this process, turning to Cross & Joftus for 
assistance in structuring their assessments and training the people who would rate candidates. In each of 
the three districts with which they worked, the consultants based the assessments on the district’s own 
evolving definitions of the standards and competencies it sought in leaders. District staff did the same in 
New York City when the new Principal Review rubric was completed and official in 2013-14. 

Assessment procedures in all districts as of 2014 included practical demonstrations, typically with simu- 
lated scenarios. Several assessments would occur over a day or more. Activities could include: 

Role play: the candidate takes the role of principal responding to a difficult semi-scripted scenario, 
such as addressing an angry parent or faculty member. 

In-basket exercise: the candidate prioritizes and addresses a set of tasks and messages that might 
arrive in an in-basket. 

School data review: the candidate receives a package of school data to analyze, discuss, and ad- 
dress in recommendations for data-informed improvement. 

Teacher observation and feedback: the candidate observes a video of teacher instruction and pro- 
vides written or oral feedback to the teacher. 


4 Our survey data do not specifically identify the time(s) when a particular respondent applied to a talent pool or applied for a principalship. We use the time of 
becoming a principal (i.e., the earliest two cohorts vs. the latest two cohorts) as a rough proxy for the time when groups of respondents experienced the hiring process. 



In interviews, district officials expressed enthusiasm about the power of these new procedures to expose 
candidates’ strengths and weaknesses. A Charlotte-Mecklenburg interviewee involved in the process said: 
“What we're doing now that we weren’t doing [a few years ago] is the role play that brings to the surface a 
lot that we would not see in a normal interview, the back and forth exchange.” 

New York City’s new process for individuals in the Principal Candidate Pool included performance tasks 
for each of three three-hour assessments that typically occurred over a six-week period. Applicants received 
one-page scenarios, and in response to each scenario they might be required to outline a plan, write a letter 
to stakeholders, or deliver a short oral presentation. The scenarios had been crafted to reveal whether can- 
didate responses reflected the skills and behaviors that the district standards called for. 

Candidates’ performance in each district was rated by teams of trained raters, including principal supervi- 
sors, central office staff from various divisions (e.g., administration, human resources, curriculum and in- 
struction), principals, and coaches or mentors. Raters used standard instruments, such as rubrics and 
evidence guides, based on the district’s leader standards. Thus, not only did the standards provide a frame- 
work for designing an assessment instrument, but language from the standards documents appeared in the 
descriptions of evidence that would signal different levels of performance. 

A Charlotte-Mecklenburg official pointed out that the current assessment procedures for the talent pool 
could generate useful feedback for candidates because of the alignment to standards. Moreover, the princi- 
pal supervisors were being coached to give feedback in terms of the competencies that the candidate still 
needed to develop: 

So we don’t say, “You did poorly in the role play, and you need to learn in the role play to do blah-blah- 
blah.” Because we’re not coaching to the activity. We coach to the competency. 


Issues of Staff Capacity for the Rating Process 

Full results of the new hiring procedures will take some time to emerge, as new cohorts of school leaders 
emerge from the process and their on-the-job performance can be seen. At this early stage, interviewees re- 
ported that performance assessments appeared to provide a more authentic understanding of candidates’ 
competencies. They observed, however, that the time demands on district staff reviewing and rating per- 
formance were substantial. The capacity issue here was not so much skill as the sheer amount of time that 
the ambitious new systems demanded. Carrying out performance assessments required substantial time on 
the part of raters and hiring managers, raising questions about sustainability. As the new systems started 
up, districts leaned on staff in ways that could lead to burnout or detract from other responsibilities. 

In Charlotte-Mecklenburg, for example, several senior district staff members attended each talent-pool can- 
didate’s multi-hour session of performance assessments and interviews. If there are a hundred applicants 
over the next two years, the person-hours will add up. In New York City, a small and dedicated central-of- 
fice staff and willing principals were rating the performance of principal applicants, and the strain associ- 
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ated with the time commitment was clear. As one participant in Charlotte-Mecklenburg put it, “It has been 
very time-consuming.... I’m not going to sugar-coat that. We’re going to see if all that time commitment is 
worth it.” 


Applicants' Perceptions of the Selection Process 

As might be expected from individuals who had recently been hired, novice principals gave relatively posi- 
tive responses to questions about the hiring process. Feedback to candidates was an area in which districts 
might be able to improve, however. The 2014 survey had a question about the process of application to a 
talent pool (or, in Gwinnett and Hillsborough Counties, application to the principal preparation programs 
that served the function of a talent pool). A parallel question asked about a process in which the respon- 
dent had applied for specific school vacancies (a process that did not take place in all cases, since districts 
might place candidates based on data they already had in hand, without requiring formal applications from 
the candidates they identified). 

At each stage, more than 80 percent of the principals reported that they had had a good opportunity to 
show their skills and that standards and expectations were clearly communicated (Exhibits 12 and 13). 
About 60 percent reported having received useful feedback, with only about 20 percent strongly agreeing 
that the feedback had been useful at each stage. 

Changes in procedures were creating some stresses for candidates. In focus groups with novice principals 
and aspiring principals in 2014, the effects of changing district policies and practices could be discerned in 
some participants’ frustration. They felt that the process lacked transparency; they were unsure of the ra- 
tionale for the changes they had heard about or experienced; and some suspected that the process had un- 
derlying unfairness. This feeling was not universal, and the focus groups were not representative of all new 
principals or aspiring principals, but the comments signaled that a sense of anxiety and suspicion may arise 
when high-stakes procedures are undergoing rapid change, and they may point to a continuing need for the 
districts to provide ample information about their hiring policies. 



Exhibit 12: Principal perceptions of their past experience 
as a candidate for a hiring pool or analogous program, 2014 


I had a good opportunity to show my skills 

Standards and expectations for my 
performance were consistently and clearly 
communicated before and during the process 

I received feedback that was useful 
for purposes of informing 
my professional practice 

I received feedback that was useful 
for purposes of identifying/defining 
my professional development needs 


I 



0 20 40 60 80 


Percentage of principals who agreed or strongly agreed (N=434) 


Exhibit reads: Thirty-two percent of first-, second-, and third-year principals in the six districts strongly agreed and 56 percent agreed that they had a good 
opportunity to show their skills as a candidate fora hiring pool or analogous program. 

Source: Principal Survey for “Evaluation of the Principal Pipeline Initiative,” 2014. 


Exhibit 13: Principal perceptions of their past experience 
as a candidate for principal positions, 201 4 


I had a good opportunity to show my skills 

Standards and expectations for my 
performance were consistently and clearly 
communicated before and during the process 

I received feedback that was useful 
for purposes of informing 
my professional practice 

I received feedback that was useful 
for purposes of identifying/defining 
my professional development needs 


I 



0 20 40 60 80 


Percentage of principals who agreed or strongly agreed (N=385) 


Exhibit reads: Twenty-seven percent of first-, second-, and third-year principals in the six districts strongly agreed and 61 percent agreed that they had a good 
opportunity to show their skills as a candidate for principal positons. 


Source: Principal Survey for “Evaluation of the Principal Pipeline Initiative,” 2014. 
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Succession Planning and Placement 


The specific procedures for filling an existing or upcoming school vacancy were revised at least once in 
each of the participating districts, and no one ruled out further changes. Denver’s changes, for example, il- 
lustrate a shift from a decentralized, ad hoc process to a more centralized one. In 2011-12, the process was 
in the hands of the principal supervisor. A principal supervisor for a group of schools took the lead in invit- 
ing applications for each vacant or soon-to-be-vacant principal position, identifying the specific criteria 
sought in that principal. Applications would then arrive from interested candidates who saw the job post- 
ing. As a principal supervisor described it in 2012, especially with reference to struggling schools, the effort 
was considerable and the results were often disappointing: 

We get some support from [the human resources office], but typically we post the position, wait for 
candidates to apply, pull the applications. You’d hope for thirty or forty, but we [usually] get eight 
to fifteen applicants, and half of them don’t meet the criteria. That’s the biggest challenge: finding 
quality candidates that have done this work before. 

Districtwide, too, there was dissatisfaction with the effort and the results. Time was spent in screening un- 
qualified applicants; effort was duplicated in multiple, school-by-school selection processes; and it might 
become clear in retrospect that the candidate placed as principal for one school would have been a better fit 
for a different school that happened to post its vacancy later. 

One change in Denver was the initiation of the principal pool in 2012-13. Another was a shift to longer- 
range succession planning at the top levels of the district in 2013-14. By spring 2014, district interviewees 
could describe how those two processes intersected for a more orderly and better informed process than 
had been possible in earlier years. In the fall, top leaders and the principal supervisors conferred to begin 
projecting an overall number of vacancies and identifying specific vacancies that might be anticipated based 
on what the principal supervisor knew about principals’ plans. They also discussed assistant principals and 
others who might be encouraged to seek leadership roles or further preparation. For the principal pool, a 
rolling process of application review had resulted in 54 candidates selected into the pool by mid-April. As 
vacancies occurred, some were quickly filled from the pool. Looking ahead in April, interviewees were gen- 
erally confident that they had a good range of candidates ready for vacancies likely to arise later. 

Hillsborough County was moving in 2013-14 toward a system of succession planning as well. In two meet- 
ings so far, overall projections and specific likely vacancies were assessed in relation to the pool of candi- 
dates who had been through the district’s principal preparation program. A participant said: 

To have these conversations, ... the succession planning meetings, to look at vacancies for the 
whole year, to talk about “these are the types of leaders needed,” to actually pull up candidates’ 
data, create lists for the [principal supervisors] — all of that was new this school year that we’re 
very excited about. 



Another participant in that district echoed the enthusiasm about the opportunity to 

...look at school needs [and] look at the skills and competencies [of individuals] coming out of the 
program and their past experiences, generating a group of candidates that would likely be success- 
ful, by matching their competencies with the needs of the school. 

Prince George’s County, which overhauled its hiring process in 2012, made other changes over the subse- 
quent years and was considering still others in 2014. The district had moved from posting specific vacan- 
cies individually to a batched process of inviting applications and, to the extent possible, filling vacancies 
from the resulting eligible pool. As of spring 2014, however, the district was planning a shift to more use of 
“school-specific vacancies” and a streamlined selection process because some challenges had become appar- 
ent. Applications had been invited for two schools whose specialized foci did not have good matches in the 
existing pool, and, more broadly, district leaders were wondering about the wisdom of excluding strong 
candidates who happened to miss the “all-call” for applications. This example suggests that any of the dis- 
tricts may seek further refinement of their succession planning and hiring procedures before the Principal 
Pipeline Initiative ends. 


Use of Data in Principal Placement 

District leaders welcomed the idea of using more data, gathered in more ways, in the hiring and placement 
process. They particularly appreciated the value of data about candidates in making the right match between 
candidates and particular schools. By 2014, selection into the talent pool was becoming close to synonymous 
with readiness for placement in a principalship. And, as districts looked ahead more systematically to the ex- 
pected principal vacancies, the hiring process was increasingly a matching process, in which the district tried 
to find the best fit for each of several schools from among interested candidates in the pool. 

The six districts were able to use school-level data to inform matching. Prince George’s County, for exam- 
ple, previously collected input informally from school community representatives at the time of hiring a 
principal, but starting in 2011 the district developed and maintained profiles of all schools that could be 
used in the hiring process. Using surveys along with achievement and other data, the district compiled a 
profile that would be relevant to the school’s leadership needs, so that a principal candidate could be 
matched to any school where a vacancy arose. 

District leaders were interested in various kinds of data about candidates for hiring purposes. They could 
readily access the data generated by their assessment process for a talent pool or district-run preparation pro- 
gram, data that were often housed in the same district office involved in principal hiring. They were also inter- 
ested in data that had traditionally been collected by different divisions and stored in separate databases, 
including data on student performance in schools or classrooms where a candidate had previously served. 

In this regard, district leaders appreciated the Principal Pipeline Initiative specifications — and the accompa- 
nying support — for development of a Leader Tracking System capturing individual, longitudinal data on 
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experience, performance, and assessed competencies. The data were not yet seamlessly available for the hir- 
ing process in spring 2014, but Hillsborough County had begun to use electronic fact sheets on potential 
candidates, and leaders in all districts looked forward to the routine use of the data in user-friendly for- 
mats. As the Leader Tracking Systems become operational, districts were expecting that data for hiring de- 
cisions would become easily accessible. 

Examples of data that districts expected to use, and in some cases had begun using, for hiring included: rel- 
evant student achievement data as a teacher and assistant principal; evaluation scores as a teacher and as- 
sistant principal; experience with certain types of schools and students; language skills; performance on 
selection and exit assessments in preparation programs; and measured competencies related to instruction 
and leadership. Thus, for example, district leaders said in interviews that they might look for a candidate 
who had experience in particular grade levels or with English language learners and whose measured com- 
petencies matched the needs that they saw in the school. None claimed that matching was becoming an 
exact science, but they welcomed an easier process of spotting candidates whose backgrounds they consid- 
ered promising matches for a school’s most visible needs. 

Much work had to take place in order for individual data to be accessible. Designing and populating each 
Leader Tracking System took time and required the expertise of specialists whom the districts employed or 
retained as consultants, as well as assistance and expert review from consultants engaged by The Wallace 
Foundation. In each district there were problems to be solved in pulling the data together and designing 
user interfaces. In New York City, for example, retrieving data from multiple data systems was a challenge. 
District information-technology staff also spoke of “being able to talk across the user and technical per- 
spectives” in the process of finding and organizing the most relevant data to address users’ needs. “When 
they say what they want,” a staff member said of the prospective users of the new system, “there may be a 
hundred decisions [still needed] about what they really want.” The system designers conducted focus 
groups with prospective users and also ensured that top district leaders were involved in all key decisions. 

A top leader in another district who had begun to see the early products of this kind of work said: 

I think the behind-the-wall technology development has been extremely helpful. It allows us to — 

with ease — pull out those people who are in the pipeline who will be good matches for the schools. 

It’s very, very helpful. 


Human Judgment as Part of the Process 

Although they were stepping up the use of systematically gathered data in hiring decisions, district leaders 
also explained that human judgment would continue to be part of the decision process, especially in match- 
ing candidates’ strengths to particular school positions. They did not expect to find a formula that would 
drive data-based decisions but, rather, knew that decision makers would use judgment in interpreting the 
available data. 



They also expected to use insights drawn from acquaintance with aspiring leaders. The district leaders val- 
ued their chance to observe prospective leaders in the district-run preparation programs in five of the dis- 
tricts, sometimes over multiple years. They knew they had to figure out how to incorporate their 
observations into selection in a responsible way, though. They recognized that their newer, systematically 
gathered data on experiences and measured skills offered a needed alternative to the kinds of informal im- 
pressions that had prevailed in the “old-boy network”: 

It’s a mindset shift, it’s a culture shift. I think that in the past people just knew who they knew, and 
they did interviews and walkthroughs and had relationships, and that's how they made decisions. 
We’re trying to get them to now also look at this really good data that shows how people per- 
formed on assessments aligned to the expectations that people are going to be held accountable for 
once they become principals, and to really think about what does this tell about whether this per- 
son is going to be the right fit for the role generally and for this school specifically. 


Hiring Assistant Principals 


Across all districts, principals customarily made the decisions about hiring assistant principals for their 
schools. Districts were beginning to give principals additional data about candidates, however, and 
Charlotte-Mecklenburg, Denver, and Hillsborough County selectively identified candidates eligible to be- 
come assistant principals. 

A Denver official explained that the data gathered in screening process for assistant principal candidates 
were “almost exactly the same” as for principals, including a paper screen, interviews, and scenario-based 
assessments. One small exception was that fewer scenarios were used with aspiring assistant principals than 
with aspiring principals. The school principal led and shaped the school-level process of reviewing specific 
candidates from the assistant principal pool, however. 
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In Hillsborough County, the principal supervisors officially selected an assistant principal from a list of three 
finalists who had completed a selective preparation program in the district and whom a principal had identi- 
fied on the basis of interviews. In most cases, though, the principal supervisors’ involvement was something 
of a formality; one interviewee estimated that the principal’s choice prevailed in “99 percent” of cases. 

Gwinnett County also tried to infuse the wealth of data available at the district level into a process of hiring 
assistant principals, although it was still a process in which principals retained the decision-making author- 
ity. Rather than accepting a principal’s comment, “I kind of like so-and-so,” district officials would ask 
about the school’s specific needs and then identify likely matches among the candidates whom they had as- 
sessed and worked with in the selective preparation program for assistant principals. As one described it: 

Principals hire for their schools, so the principals contact us for suggestions. There’s a process that 
we have in place through screening. There’s a list of available applicants. ... What we do instead of 
trying to make the decisions for them is just to provide as much information and evidence as possi- 
ble so that they can make that best decision when hiring for their schools. 
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Remaining Issues in Hiring and Placement 


District officials said they were pleased with the changes they had made in hiring procedures. Interviewees 
in each district spoke highly of the principal candidates they were seeing and the matches they were making 
between candidates and schools. Still, the ways in which they were using data were new and unproven. 

And they acknowledged challenges: finding leaders for some types of schools remained difficult; and their 
assessment procedures did not yet bring to light all the needed types of information about candidates. 

Several kinds of schools continued to pose challenges for leadership selection. It was common for high 
schools to draw too few strong candidates. Leaders in all districts also acknowledged challenges in finding 
the right candidates for high-need schools. These could include schools with low student achievement or 
low family incomes, or schools in which parents of high socioeconomic status made many demands on the 
principal. District leaders in Charlotte-Mecklenburg and Hillsborough County said they had seen both 
high-achieving and low-achieving schools pose leadership challenges, and that they still did not have as 
many strong candidates as desired when a vacancy arose in a school of either kind. 

Considering the strengths and weaknesses they had seen in the work of their newest principals, interviewees 
made some similar observations across all districts except New York City (where no one wanted to general- 
ize across a cohort of 200 new principals annually). Leaders in the other five districts praised the new prin- 
cipals’ deep knowledge of instruction. However, they expressed concern about some principals’ difficulties 
in working with the school community. In the following comments, from three different districts, veteran 
leaders described this recurring area of weakness that they had seen in new, highly prepared principals: 

They’re very passionate people. They get into the school ready and passionate to make a change, 
and it’s like they’re coming in with a sledgehammer and making change, but . . . they don’t realize — 
it’s personal leadership skills, they’re missing the boat. No one’s following them if they’re going in 
with a sledgehammer. So how do they navigate through that politically?... Are there ways they can 
navigate it better so it’s not as disruptive a change? 

How do you use your positional authority? How do you read the culture of a school in terms of, 
say, this is not a turnaround school if we’re sitting at 97 percent proficiency? Perhaps there are 
many things you need to work on, but we don’t need to approach this like it’s a turnaround situa- 
tion in your first principalship out of the gate. ... I would say more of our principals struggle . . . 
over their human relations and inability to get the work done through and with people. 

I think sometimes you have to remind them about relationship building. They go in, and sometimes 
they go head on, tackle things. They have to remember they’re not part of the family yet. 

With this concern in mind, district leaders said they wished they had better ways to measure emotional in- 
telligence (the term used in Denver) or “micro-political” skills (the term used in Charlotte-Mecklenburg) 
and to bolster aspiring principals’ capacity to work effectively with a faculty, staff, and community. In 
Gwinnett County, a district leader described the use of a promising tool: 



This year, [we used a] pilot disposition assessment which was quite enlightening. We found things 
about individuals that would derail them if they’re not aware that they need to manage that part of 
their personalities. [Some who are] not touch-feely — now we know this, and they know it, and 
they’re getting support to make them more aware of the messages they send. ... [In the past we’ve 
been] missing some pieces that get at ... the emotional wake a person will leave if they don’t ad- 
dress those areas. 


Summary: Changes under the Initiative , and a Look Ahead 

During the grant period, all six districts revised the hiring process with further development and specifica- 
tion of hiring stages, criteria, and roles. These changes added more screening decisions and discussion prior 
to taking final candidates to the superintendent. Changes ranged from fine-tuning pieces of the process, 
such as selection criteria, to the development of entirely new components, such talent pools and data-based 
procedures for matching candidates to positions. The results were hiring processes that were more system- 
atic, deliberate, and standards-based. 

Every district had a new or modified screening step producing a pool of candidates who were al- 
lowed to apply for leadership positions. Overall, the districts moved closer to winnowing candi- 
dates to those with a good chance of fitting into a position within the next year or two. In New 
York City, a newly designed step provided ratings on standards-based competencies for all candi- 
dates in the hiring pool, which hiring managers could use. In Gwinnett and Hillsborough Counties, 
districts that had instituted sequences of two selective internal programs, the selection stages were 
now beginning years before a candidate might be considered for a principal position. 

Every district developed or modified selection criteria or instruments aligned with standards and 
specifying behaviors or other evidence for each criterion. Selection stages, whether into a pool or in 
hiring, were expected to incorporate the uniform use of rubrics, tasks, and interview questions 
aligned with standards. The districts invested in design and development efforts to identify priority 
competencies for each stage and provide tools and training for consistent ratings. 

Every district was using data on candidates and schools in more systematic ways to match candi- 
dates to specific school positions. Districts were adding procedures for considering data collabora- 
tively as a group of senior managers. In Denver, Gwinnett County, and Hillsborough County, 
individual supervisors had lost some of their independence in winnowing to the finalists for a posi- 
tion. 

Denver and Hillsborough County introduced new procedures by which top district leaders held 
succession-planning discussions, reviewing data-based projections of principal vacancies and work- 
ing toward agreement on their priorities for leader selection in the short term and leader develop- 
ment in the longer term. (Charlotte-Mecklenburg and Gwinnett County had similar planning 
processes in place before the initiative). 


Policy Studies Associates, Inc. 



Districts Taking Charge of the Principal Pipeline 


Finally, there was a noticeable sense of excitement in the districts around what officials saw as improve- 
ments in the hiring process, both those already accomplished and those that could follow from added selec- 
tion stages and more readily available data. 
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6. Evaluation and Support 


Every district worked on every component of the 
Principal Pipeline Initiative design in every year, but 
some staging of their work was evident. The component 
that included leader evaluation and professional sup- 
port was on a slightly slower timeline than the others. 
This component was scheduled to be solidified in 2015. 

Districts encountered the limits of their existing capac- 
ity for leader evaluation and support in their initial 
work on these functions. They worked to build capacity 
while they iteratively piloted, revised, scaled, and re- 
fined new procedures for evaluation and support. We 
begin the chapter with an overview of the types of ca- 
pacity available in the districts and from their partner 
organizations, then analyze the work done to strengthen 
leader evaluation and support from 2011 to 2014. 


Capacity for Evaluation and Support 


Takeaways 

1. Districts added capacity through positions 
and training for supervisors, mentors, or 
coaches, trying to give new principals a 
larger amount of skilled support. 

2. Developing evaluation criteria aligned 
with standards required multiple rounds of 
clarifying the criteria and preparing the 
supervisors to use them. 

3. Principals tended to see mentors or 
coaches as more helpful than supervisors, 
although both groups received positive 
ratings. 

4. Data on individual strengths and 
weaknesses could support greater tailoring 
of professional development, but work 
remained to be done to realize this vision 
for principals and assistant principals. 


Districts’ capacity for principal evaluation and support 
resided in several different roles. The district staff who 
directly supervised principals took a major role in eval- 
uation and were also expected to support principals. Support also came from mentors or coaches, from the 
central offices in charge of professional development, and from external partners. 
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Principal Supervisors 

The participating districts put their principal supervisors in a dual role of both evaluating and supporting 
principals. New York City was a partial exception, with an anomalous arrangement in which formal re- 
sponsibilities for supervision and support were assigned to two different groups. By state law, community 
superintendents or high-school superintendents served as formal rating officers of principals. The clusters 
and networks, created as a school support structure independent of the geographically based superinten- 
dencies, were expected to give schools and their principals feedback and support. Still, the New York City 
principal supervisors whom we interviewed also gave numerous specific examples of ways in which they 
tried to create learning opportunities for principals and assistant principals whom they supervised. 
Supervision arrangements in the other districts were also redesigned for a greater focus on instructional is- 
sues and on building principals’ capacity. 
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A study of the principal supervisor role that The Wallace Foundation commissioned from the Council of the 
Great City Schools was influential in this initiative (Corcoran, Casserly, Price-Baugh, Walston, Hall, & Simon, 
2013). Based on a national survey and on site visits to the Principal Pipeline Initiative districts, the study’s rec- 
ommendations for any sizable district included increased staffing, clearer goals, professional development and 
coaching, and clearer lines of both communication and accountability for principal supervisors. All of the dis- 
tricts in the Principal Pipeline Initiative subsequently made plans to enlarge their cadres of supervisors — thus re- 
ducing each supervisor’s caseload of principals — and to continue clarifying and supporting the supervisor’s role. 
Denver added enough principal supervisors in 2012-13 to reduce each supervisor’s caseload to ten principals. 

Early in the initiative, district leaders had seen tension in the shift that they were asking their principal supervi- 
sors to make, from hands-on solvers of administrative problems to coaches strengthening instructional leader- 
ship. As of 2014, no district leader claimed that the tension between administrative and instructional emphases 
had gone away. A leader in one district said of the shift in role that the principal supervisors were experiencing: 

It has been a slow but steady process. You can see the shifting of the mindsets in terms of where 
their priorities are, how they look at the big picture now. . . . [But] trying to change the mindset 
sometimes, or the way of work, ... [can be] difficult. It’s almost like a rubber band. You stretch it, 
but as soon as you let go it goes right back. 
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Capacity for Mentoring, Coaching, and Professional Development from Inside and 
Outside the District 

Each district employed or had contracts with individuals who provided support for novice principals. For 
example, novice principals in Denver had two distinct types of support: “mentors” who were experienced 
principals in the district and helped with whatever problems arose on the job; and “executive coaches” on 
contract with the district who worked with them on the process of clarifying and pursuing their profes- 
sional goals in leadership. 


Charlotte-Mecklenburg organized its principal support into a multi-year sequence. The district had started 
its year-by-year program of induction support before the initiative began, and it used initiative support to 
extend the program to a full five years. The design illustrates the interplay of internal and external sources 
of support for novice principals: 


First- and second-year principals were matched with a Consultant Coach who met with and ad- 
vised principals in small groups. 

Beginning in 2011-12, second-year principals participated in the National SAM Innovation Project. 
SAM coaches worked with principals on time use and instructional leadership, and by 2014, SAM 
coaching was conducted by district staff members. 

Third-year principals participated in the Executive Leadership Institute at Queens University of 
Charlotte. A collaboration between the university’s education and business schools, the program 
trained principals on leadership styles and organizational change. 



Fourth-year principals participated in the Innovation Institute of the nonprofit McColl Center for 
Art and Innovation, which focused on innovation for leaders from a variety of backgrounds. 

A fifth-year capstone experience through the Center for Intentional Leadership rounded out the 
support sequence. 

Across districts, external partners also supported principals. The National SAM Innovation Project worked 
with principals in Charlotte-Mecklenburg, as described above, and also in the other districts. The New 
York City Leadership Academy had had a contract to provide coaching to all first-year principals in New 
York City since well before the initiative. New Leaders provided ongoing coaching to its graduates after 
they became principals in Charlotte-Mecklenburg, New York City, and Prince George’s County. The School 
Leaders Network helped organize and facilitate peer learning for new principals and assistant principals in 
New York City and Prince George’s County. 

Another role for partner organizations was that of building district staff members’ capacity to support new 
principals. The New York City Leadership Academy worked with Denver to support mentoring training, 
for example, and it also built capacity in Charlotte-Mecklenburg’s partnerships for preparation and induc- 
tion. New or expanded partnerships for novice-principal support in three districts grew out of the districts’ 
and partners’ previous experience working together: 

Charlotte-Mecklenburg received help with the coaching process from two universities. The director 
of a partner preservice program at Winthrop University also had a major role in helping the district 
develop a coaching program and provided training for the district’s coaches. Queens University 
started a three-day academy for the district’s leadership coaches. 

Hillsborough County contracted with the New Teacher Center to help develop a principal coach- 
ing program, train the coaches, and work with principal supervisors to help build their skills in 
leader development. Later, Cross & Joftus was engaged to build capacity for principal evaluation 
aligned with the district’s leader competencies. 

Under the grant, Prince George’s County expanded the participation of mentor principals in the 
National Association of Elementary School Principals’ Leadership Immersion Institute. Mentor 
principals won special certification by participating in training and in a nine-month internship pro- 
gram in which they practiced providing feedback and support to new administrators. 


Leader Evaluation 

Each district came to the initiative with a history of evaluating principals’ performance, but each revisited 
its procedures under the initiative. Newly articulated district leader standards, as discussed in Chapter 3, 
were at the core of this process. State education agencies also played a part; for the five states (all but 
Colorado) that were grantees under the federal Race to the Top program, establishing criteria for leader 
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evaluation was part of the work for which the state was held accountable by the U.S. Department of 
Education. Gwinnett County and Hillsborough County, which had evaluation systems for leaders before 
their states did, were able to proceed in fine-tuning their own systems rather than taking on entirely new 
state-mandated systems. Prince George’s County submitted its new evaluation system to the state for ap- 
proval in 2013 and was asked to make only minor changes. Charlotte-Mecklenburg was required to use the 
state’s evaluation criteria but also developed evaluation rubrics that reflected local priorities as well as state 
priorities. In New York City, as described in detail in Chapter 3, the local Quality Review Rubric was the 
source for a new principal evaluation for which the district succeeded in winning state approval. 


The Process of Developing Standards-Based Evaluation 

Principal evaluation in every district in 2014 included measures of student performance and also ratings 
that supervisors gave in relation to a set of standards-based evaluation criteria. The criteria for supervisors’ 
ratings were evolving. For example, Gwinnett County was using its longstanding Results-Based Evaluation 
System, in which one major component continued to be school-level performance benchmarked against 
comparable schools in other urban districts nationwide. The district was beginning to use new procedures 
and criteria for supervisor ratings, however, with the intention of phasing in a new system for that compo- 
nent of the evaluation system. 

At the request of The Wallace Foundation, five of the districts participating in the initiative had undertaken 
to use the VAL-ED instrument, developed (with Wallace support) to capture evidence on research-based ele- 
ments of principal effectiveness (Goldring, Porter, Murphy, Elliott, & Cravens, 2009; Murphy, Elliott, 
Goldring, & Porter, 2007). This instrument gathers ratings in a 360° process from the principal, the super- 
visor, and teachers. (The sixth district, Denver, used its own 360° instrument instead of VAL-ED.) Four dis- 
tricts used VAL-ED on a pilot basis or only as a source of formative feedback to leaders, but Hillsborough 
County used it districtwide as part of its consequential evaluation for principals. 

Some interviewees commented that 360° ratings provided valuable insights in assessing principals’ perform- 
ance. In both Charlotte-Mecklenburg and Denver, principal supervisors reported instances in which they 
had recommended dismissal of a principal after learning that he or she lacked teachers’ support or respect. 
Their assessments had relied on additional information, beyond a survey, but they cited this experience in 
endorsing the collection of data from teachers. As one of the supervisors said: 

We had to dismiss a principal last year, and I will say, teacher perception surveys often play a big role 
in that, because if your teachers have absolutely no confidence in you, how can you run a school? 

Charlotte-Mecklenburg, Denver, Gwinnett County, and Hillsborough County reported doing extensive 
work to prepare principal supervisors to use new evaluation instruments. The process of learning how best 
to operationalize new standards in evaluation instruments was also informing refinement of the indicators 
that accompanied the standards — a process that was far from finished in spring 2014. A Denver official de- 
scribed how principal supervisors were carrying out one step in beginning to calibrate their ratings on indi- 
cators and the associated rubrics: 



We went through all the different indicators and said, what data do you look at? For each of those 
indicators, what would be the kinds of data and evidence that you're collecting? And then we said, 
okay, that is a ton because there were charts all around the room full of stuff. So then we said, let’s 
narrow it and look across these, where do we see commonality? We call those high-leverage data 
points. We listed those out and then spent the next session actually looking at, okay, how do you 
collect your data? 

A Gwinnett County official also emphasized the careful work being done in the process of calibration: 

In the new principal evaluation system .... [we are] now watching the team [of principal supervisors] 
really look at wrestling through these standards, and how you evaluate them in a consistent format. 


Assistant Principal Evaluation 

For the most part, assistant principal evaluation instruments mirrored those used for principal evaluation. 
In Charlotte-Mecklenburg, the state-mandated evaluation instrument was identical for principals and assis- 
tant principals. In three other districts, the instrument was at least lightly adapted; in Prince George’s 
County, for example, the indicator language for standards was reframed with words such as “supports the 
principal in...” each leadership function. Gwinnett County had similar minor adaptations in the assistant 
principal evaluation. Denver had an “evidence guide” for rating assistant principals that differed from the 
one used with principals, although the overall expectations for the two positions were aligned with each 
other. Hillsborough County used its own 360° instrument, aligned with the district’s leader competencies, 
for assistant principals. A district official commented that the results gave principals useful information 
about the competencies in which their assistant principals needed further development. The major excep- 
tion was in New York City, which had neither formal standards nor an evaluation instrument for assistant 
principals. 

Just as principals had the final say on hiring assistant principals, they held the reins in evaluating assistant 
principals. Some district officials expressed concern about the way principals carried out these evaluations, 
commenting that principals should more assertively communicate negative results to their assistant princi- 
pals. This was related to their concern that some marginal performers were occupying assistant principal 
positions and thus were blocking opportunities for aspiring leaders with higher potential. 


Perceptions of Evaluation among Principals and Assistant Principals 

Survey respondents expressed mildly positive views about their experience with leader evaluation (Exhibits 
14 and 15). Readers should note that these findings pertain to the evaluations conducted in 2012 and 2013. 
We asked about the previous school year’s evaluation because our surveys were fielded in early spring each 
year, before the current year’s full cycle of evaluation and feedback was likely to have been completed. 

There were no significant differences in responses across years of the survey (i.e., in the responses pertain- 
ing to evaluation in 2012 vs. 2013). 
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Exhibit 14: 

Principal perceptions of their 2012 and 2013 performance evaluation 


The performance evaluation system: 

Percent of principals 
saying to a considerable 
or great extent of their 
2012 evaluation 

(N= 204 ) 

Percent of principals 
saying to a considerable 
or great extent of their 
2013 evaluation 

(N=259) 

a. Was useful for purposes of informing my professional practice 

65% 

54% 

b. Provided clear expectations for my performance that were 
consistently and clearly communicated 

64 

58 

c. Adequately/accurately reflected my performance 

63 

54 

d. Was useful for purposes of identifying/defining my 
professional development needs 

58 

50 

e. Captured the breadth and complexity of my leadership role 

58 

49 


Exhibit reads: In 2013, 65 percent of first-, second-, and third-year principals in the six districts reported that their 2012 performance evaluation was useful for purposes of informing 
their professional practice to a considerable extent or to a great extent. In 2014, 54 percent of first-, second-, and third-year principals in the six districts reported that their 2013 
performance evaluation was useful for purposes of informing their professional practice to a considerable extent or to a great extent. 

Source: Principal Survey for “Evaluation of the Principal Pipeline Initiative,” 2013 and 2014. 

60 


Exhibit 15: Assistant principal perceptions of their 
2012 and 2013 performance evaluation 

The performance evaluation system: 

Percent of APs saying to 
a considerable 
or great extent of their 
2012 evaluation 

(N=i 32 ) 

Percent of APs saying to 
a considerable 
or great extent of their 
2013 evaluation 

(N=243) 


a. Adequately/accurately reflected my performance 

69% 

59% 

b. Was useful for purposes of informing my professional practice 

64 

60 

c. Provided clear expectations for my performance that were 
consistently and clearly communicated 

62 

56 

d. Was useful for purposes of identifying/defining my 
professional development needs 

64 

56 

e. Captured the breadth and complexity of my leadership role 

59 

51 


Exhibit reads: In 2013, 69 percent of first-, second-, and third-year assistant principals in the six districts reported that their 2012 performance evaluation adequately/accurately 
reflected their performance to a considerable extent or to a great extent. In 2014, 59 percent of first-, second-, and third-year assistant principals in the six districts reported that their 
2013 performance evaluation adequately/accurately reflected their performance to a considerable extent or to a great extent. 

Source: Assistant Principal Survey for “Evaluation of the Principal Pipeline Initiative,” 2013 and 2014. 









Most principals (77 percent) received their evaluation results in written form, and most (64 percent) had a 
formal meeting with their evaluator to receive feedback (Exhibit 16). About one-quarter of responding 
principals had received feedback in an informal meeting with their evaluator. Among the principals who 
had a mentor or coach, between a quarter and a third received evaluation feedback in an informal meeting 
with that person. These responses did not change significantly between years. 


Exhibit 16: Feedback received by principals on the results of their 
2012 and 2013 performance evaluation 


Did you receive formal and/or informal feedback on 

Percent of respondents 
who were principals in 

Percent of respondents 
who were principals in 

the results of your performance evaluation for 

2011-12 

2012-13 

LAST school year? 

(N= 204 ) 

(N=259) 


a. Yes, written report 

77% 

77% 

b. Yes, required formal meeting with my evaluator 

64 

64 

c. Yes, informal meeting with my mentor/coach 

26 

30 

d. Yes, informal meeting with my evaluator 

24 

26 

e. No, 1 received no feedback on my evaluation 

5 

3 

f. Other 

1 

1 


Exhibit reads: In both 2013 and 2014, 77 percent of second- and third-year principals across the six districts reported that they received a written report summarizing the results of their 
evaluation in the previous school year. 

Note: Item cwas calculated only for those respondents who reported that they had a mentor/coach (in 2013, N= 129; in 2014, N= 166). 

Source: Principal Survey for “Evaluation of the Principal Pipeline Initiative,” 2013 and 2014. 


An example of the positive views of the evaluation process that some principals reported in surveys and 
focus groups came from a principal in Charlotte-Mecklenburg, who highlighted the value of informal com- 
munication. This principal said that the evaluation instrument itself needed improvement. However, she 
emphasized that recurring, informal interaction with a supervisor provided useful evaluative feedback 
grounded in an understanding of the school: 


The feedback that I get from my supervisor is ongoing. It is not, “Okay, here’s the evaluation tool, 
we’re going to sit down and have an hour’s conversation about it and that’s your feedback.” The 
feedback comes throughout meetings when I talk with him during school visits, phone calls on the 
cell phone, emails back and forth. ... he knows where I’m at, he knows where my school is, he un- 
derstands the culture and climate of my building, and he and I have had conversations about areas 
that I need to grow in and areas where I have strengths. 
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Professional Development and Tailored Support 


As of spring 2014, all districts were following the initiative’s design by making — or planning — improve- 
ments in the professional development and individual support they provided to principals. Additional sup- 
port for assistant principals was being considered. 


Support for Principals 

As described earlier in this chapter, all six districts had worked to bolster the capacity of supervisors and 
mentors or coaches to work with principals. Districts reorganized the professional development offered to 
groups of school leaders, and they continued to refine their policies and practices on coaching. Leaders in 
several districts expressed enthusiasm about the possibility of tailoring professional learning and support 
opportunities to individual principals’ identified needs. 

Charlotte-Mecklenburg was adding a new senior position in its professional development department in 
summer 2014: an executive director of leadership development whose full-time responsibility would be the 
professional learning opportunities for principals, assistant principals, and district leaders. One priority for 
the position would be professional development for those who had been selected into the talent pool; an- 
other would be bringing a more standard approach districtwide. A district leader described the work to be 
done, emphasizing the need to develop differentiated options within a consistent framework of leadership 
development: 

We have strong induction, so it’s not to take the place of that. It’s more about giving people more 
options, and trying to bring it all together and have a consistent way we develop APs and princi- 
pals from an internal perspective.... A major part of our strategic plan is continuing to invest in 
leadership development. 

In Denver, a centerpiece of support was the introduction of coaches focusing on team building and personal 
leadership with new principals. For aspiring principals in the district-run Learn to Lead program, the dis- 
trict’s standards were used to frame individual plans for professional growth, based on each participant’s 
measured strengths and weaknesses. Denver’s intended changes in professional development centered on 
“shifting from sort of ‘one size fits all’ to a more differentiated approach.” A first step in that direction was 
to offer more choice in professional development offerings, but a district leader explained in 2014 that the 
next step would be to move beyond relying on a voluntary approach and instead determine “how can we 
guide people” to the resources they needed. Like the aspiring leaders, the principals had professional 
growth plans, and the district expected to gather and use data on the principals’ progress on their plans, as 
well as on the extent to which they had taken advantage of particular professional development offerings. 

Gwinnett County reorganized its principal supervision structure in 2013-14, adding more supervisors to 
support principals as instructional leaders. The district also continued to support principals with “leader 
mentors,” who were retired principals with strong records in instructional leadership, offering coaching on 



tasks as they arose and providing guidance on leadership responsibilities. An innovation in 2013-14 was 
the use of a log to capture a record of the interactions that each leader mentor had with each principal. The 
district’s principal meetings had been reorganized, shortening the time spent on general announcements so 
as to focus more time discussing problems of practice, such as employee engagement. In Gwinnett County, 
as in Denver, there was an intention of increasingly tailoring professional development offerings. As a dis- 
trict official put it, “It’s not just, ‘we have a good seminar.’ It’s really customizing that to the principal’s 
need and level of performance. That’s very ambitious, I understand that.” 

Hillsborough County incrementally increased the number of supervisors and coaches working with princi- 
pals and provided them with coaching for their roles (as described earlier), in an effort to provide not only 
more support for principals but also support that was increasingly focused on instructional leadership. As 
of 2014, one of the changes under way was linking the supervision and coaching to professional develop- 
ment. After sessions on a particular leadership skill, the supervisor and coach would follow up, asking a 
principal how he or she planned to put the skill to work in the school. 

New York City had put support arrangements in place during the 2000s, in particular the network struc- 
ture and the coaching support for first-year principals from the New York City Leadership Academy. Under 
new district leadership in 2014, these arrangements were still in place. 

Prince George’s County also had new leadership in 2013-14. The first change introduced in principal sup- 
port was the re-introduction of universal meetings for principals, with the aim of ensuring that all received 
the same, clear messages about district priorities. At the end of the school year, additional principal supervi- 
sors were hired from the ranks of principals. 

In support, as in hiring, district leaders saw the potential of the data in the Leader Tracking System to im- 
prove the district’s work. They believed that individual leaders’ assessed competencies could and should 
drive the differentiation of support. A district-level interviewee in Charlotte-Mecklenburg sketched a vision 
that had counterparts in the other districts as well: 

Hopefully as we get our Leader Tracking System all fully implemented and developed, we’ll be able 
to take what we’re learning from our current selection process and really identify which of these 
competencies are the person’s strength, what are the areas for growth, and be able to give that data 
over to the [professional development] department so that they’ll be able to provide what's needed 
for each of those people, and giving that information to our [principal supervisors and their staff] 
so that they can work individually with the people within their groups, and even trickle that down 
to the principals so they know where to help their APs grow. 

Principals’ survey responses did not show significant changes in the overall extent of mentoring or coaching 
from 2013 to 2014. A high proportion of first-year principals in every district had mentors or coaches in 
both 2013 and 2014, according to our survey data, but districts varied in the extent to which they assigned 
mentors or coaches to principals after their first year on the job (Exhibit 17). 
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Exhibit 17: 

Percent of principals having mentors or coaches, by cohort, 2013 and 2014 


District 

2013 

2014 

First-year 

principals 

(N=112) 

Second-year 

principals 

(N=i4o) 

Third-year 

principals 

(N=8 7 ) 

First-year 

principals 

(N=20l) 

Second-year 

principals 

(N=i83) 

Third-year 

principals 

(N=i 38 ) 

Charlotte-Mecklenburg 

100% 

100% 

27% 

81% 

82% 

21% 

Denver 

87 

25 

N/A* 

88 

76 

48 

Gwinnett County 

100 

79 

29 

100 

100 

50 

Hillsborough County 

100 

100 

94 

100 

100 

N/A* 

New York City 

100** 

75** 

65** 

96 

54 

45 

Prince George's County 90 

65 

N/A* 

87 

63 

0 

OVERALL 

95 

72 

49 

93 

80 

35 


Exhibit reads: Among first-year principals in Charlotte-Mecklenburg, 100 percent reported having a coach or mentor in 2013 and 81 percent reported having a coach or mentor in 
2014. 


64 


Source: Principal Survey for “Evaluation of the Principal Pipeline Initiative,” 2013 and 2014. 
* Percentage not reported due to small N. 

**New York data should be viewed with caution due to the low response rate (31 percent). 


Both supervisors and mentors or coaches were expected to provide support to novice principals, but princi- 
pals might perceive differences in these sources of support. Therefore our survey posed two parallel ques- 
tions to principals, one asking them to rate the support they had received from their supervisor in 2013-14 
(identifying this person as “the person who evaluates you,” since the supervisors’ titles varied across and 
within districts), and the other asking the same question about support from their mentor or coach (if they 
had one). While principals gave high ratings to both groups for their knowledge and for the process and 
quality of the support they provided, statistically significant differences were found in several responses 
(Exhibit 18). 




Exhibit 18: Principal perceptions of the support received from 
principal supervisor/evaluator and mentor/coach, 201 4 


Overall, the support I received this school year has: 


Been provided by someone who is knowledgeable 
about school leadership* 

Addressed my specific needs* 

Helped me set effective goals and develop an action 
plan to reach those goals* 

Been adapted or improved based on my feedback* 

Helped me create or improve structures and 
strategies that support my teachers in using 
student data to drive instruction* 

Helped me select professional development 
that meets my needs* 


96 

89 

89 

79 

85 

75 




84 


68 





82 

1 

71 





73 


58 



0 20 40 60 80 100 


■ Support received from Mentor/Coach (N=381) 

■ Support received from Supervisor/Evaluator (N=523) 


Percentage of principals who agreed 
or strongly agreed 


65 


Exhibit reads: Ninety-six percent of first-, second-, and third-year principals in the six districts who had a mentor/coach agreed that the support they have 
received from their mentor/coach has been provided by someone who is knowledgeable in school leadership; 89 percent of first-, second-, and third-year 
principals in the six districts agreed that the support they have received from their principal supervisor/evaluator has been provided by someone who is 
knowledgeable in school leadership. 

Source: Principal Survey for “Evaluation of the Principal Pipeline Initiative,” 2014. 

*The difference for mentor/coach is significant. 


Similarly, the survey allowed us to compare principals’ reports of the extent to which each of three poten- 
tial sources of support — a mentor or coach, a supervisor, or professional development — had led them to 
make changes in their work (Exhibit 19). The individual help from a mentor or coach had the highest rate 
of strong agreement as a perceived stimulus to change in professional practice. 
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Exhibit 19: Principal agreement that different sources of support 
led them to make changes in their work, 2014 


Mentor/coach (N=381)* 



Supervisor/evaluator (N=523) 



Professional development 
experiences (N=497) 



0 


20 


40 


60 


80 


100 


■ Strongly Agree 

■ Agree 


Percentage of principals who agreed or strongly agreed 


* 


The difference for strongly agree for mentor/coach is significant 


Exhibit reads: Forty-five percent of first-, second-, and third-year principals in the six districts strongly agreed and 43 percent agreed that the support they 
received from their mentor/coach led them to make changes in their work. 

Source: Principal Survey for “Evaluation of the Principal Pipeline Initiative,” 2014. 

Support for Assistant Principals 

Formal programs of support for new assistant principals existed in Charlotte-Mecklenburg and 
Hillsborough County. Charlotte-Mecklenburg had a two-year induction program for its assistant princi- 
pals, including monthly meetings for professional development led by district staff on topics such as in- 
structional coaching and school improvement plans. The district also offered an Assistant Principal 
Academy at a nearby university designed to help the assistant principals see their opportunities for influ- 
ence as school leaders. A new district position, due to start in summer 2014, was going to focus on profes- 
sional development for principals and assistant principals and was expected to strengthen the assistant 
principal induction. A district official spoke of the importance of this work: 

I think that we’ve recognized that we could do a better job of helping with the induction of our 
new APs. Because that’s our real, best, true pipeline to the principalship, so if we spend a lot more 
time grooming and developing them it will pay off in the end. 






Hillsborough County launched a two-year induction program for assistant principals under the initiative, 
focused on individual mentoring. Each novice assistant principal was assigned a mentor who would supple- 
ment the informal mentoring that principals were expected to offer to their own new assistant principals. A 
cadre of “master mentors” helped to oversee the mentoring. 

Prince George’s County was also planning individualized professional development for assistant principals, 
while New York City recognized this as a possible area for improvement. An official in New York City said: 

There are programs for assistant principals that are designed to help them become principals but 
not [programs that say,] you just became an AP and you want to learn how to do the job well and 
you need some support. We don’t have strong centrally based programs for that. 

Rates of mentoring or coaching support reported by assistant principals varied a great deal across districts, 
with no clear trend from 2013 to 2014 (Exhibit 20). Hillsborough County stood out in both years as most 
consistently providing mentoring or coaching for assistant principals in their first two years. The other dis- 
tricts were less apt to provide mentoring or coaching support to assistant principals beyond their first year 
on the job. 


Exhibit 20: Percent of assistant principals having mentors or coaches, 

by cohort, 2013 and 2014 



District 

2013 

2014 

First-year 

principals 

(N=i 48 ) 

Second-year 

principals 

(N=i53) 

Third-year 

principals 

(N= 4 i) 

First-year APs 

(N= 285 ) 

Second-year 

APs 

@= 199 ) 

Third-year 

APs 

(N=i75) 

Charlotte-Mecklenburg 

50% 

20% 

N/A* 

26% 

28% 

21% 

Denver 

81 

39 

18 

74 

37 

29 

Gwinnett County 

52 

17 

11 

97 

57 

23 

Hillsborough County 

100 

96 

N/A* 

89 

84 

52 

New York City 

N/A** 

N/A** 

N/A** 

50 

41 

31 

Prince George's County 

43 

24 

N/A* 

19 

N/A* 

14 

OVERALL 

66 

44 

13 

58 

43 

28 


Exhibit reads: Among first-year assistant principals in Charlotte-Mecklenburg, 50 percent reported having a coach or mentor in 2013 and 26 percent reported having a coach or 
mentor in 2014. 

Source: Assistant Principal Survey for “Evaluation of the Principal Pipeline Initiative,” 2013 and 2014. 

* Percentage not reported due to small N. 

**New York data were excluded from the 2013 analysis due to low response rate. 
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Summary: Changes under the Initiative and a Look Ahead 


The districts had introduced what they believed to be promising improvements in their systems of evalua- 
tion and support for school leaders. They had begun to address the historical weakness of principal profes- 
sional development (Darling-Hammond et al., 2007). However, as of 2014 no district leader claimed that 
their evaluation and support systems were fully satisfactory — or even fully in place. Leaders in each district 
acknowledged that more work lay ahead in firming up evaluation systems and deepening professional 
learning opportunities. 

In evaluation, all districts had made changes under the initiative. New York City had evaluation criteria for 
principals that district leaders believed to be working very well due to their alignment with the Quality 
Review Rubric. Each of the other districts had begun to introduce a new or revised evaluation system but 
was continuing to fine-tune it and, in particular, to work with principal supervisors to calibrate their rat- 
ings. They expressed optimism that the alignment of evaluation with their leader standards and competen- 
cies would help them target their efforts to strengthen principals’ performance. 

Leader support intensified with the hiring of additional staff to serve as supervisors, mentors, or coaches, 
and districts continued their efforts to build the capacity of those charged with supporting principals’ in- 
structional leadership. They enlisted the help of partner organizations to deliver coaching or to work with 
district supervisors, mentors, and coaches. Mentoring or coaching was widely reported by first-year princi- 
pals but much less so by principals in their third year. The extent of mentoring and coaching for assistant 
principals varied a great deal across districts, and district leaders acknowledged that they could do better in 
the process of induction for assistant principals. 

This component of the Principal Pipeline Initiative, as expected, showed less complete implementation than 
others, as districts worked to craft better functioning systems. In particular, they were addressing the com- 
plexity of providing reliable ratings on their new evaluation criteria and organizing systems of support that 
would reach more leaders with effective, individually tailored learning opportunities. 



7. Conclusions 


T hree years into their work in the Principal Pipeline Initiative, the six districts had learned much and 
were deepening the strategic thinking that they brought to bear on strengthening their principal 
corps. They were seeing the feasibility of long-term improvements in their own policies and prac- 
tices that might resolve the problems that they had initially perceived: too few strong candidates were ap- 
plying for principalships; and too many novice principals were struggling. Their work was not finished in 
2014, and only the earliest results had emerged. Still, other districts may find useful ideas in these districts’ 
priorities and strategies. 


Several policy instruments and approaches helped the districts take charge of their opportunities to influ- 
ence individual aspiring and novice principals and, thus, to improve school leadership districtwide. 
Systematic data use was one. The districts were compiling and organizing data about individual leaders and 
aspiring leaders into tracking systems with user-friendly interfaces that could inform decisions. By looking 
systematically at data on individuals’ past performance and career progressions, they believed they could 
make better-informed decisions on hiring and placing principals. Data could also inform decisions about 
tailoring support to a particular leader’s needs and, further, organizational decisions about strengthening 
programs of preparation and support. 


The standards and competencies articulated for school leaders proved to be a more powerful policy instru- 
ment than many (including the authors of this report) had expected. Rather than sitting on shelves, the dis- 
tricts’ statements about the capabilities and performance that they expected in school leaders were put to 
work in job descriptions, in the curricula of preparation programs, and, crucially, in assessments and support 
systems for aspiring and novice principals. This work was not finished in 2014; the indicators and rubrics 
that accompanied standards were undergoing revision as district staff learned from pilot tests. At this point, 
however, the standards and competencies could be seen as fulfilling their purpose of providing a common 
language for leadership in each district. In New York City, where a careful process of standards development 
was completed and put to work in the official process of principal evaluation, district leaders saw its power 
in “making sure that hiring managers, central office, principals in the field, coaches, [and] principal prep 
programs” had a shared understanding of the work for which principals were held accountable. 


In principal preparation, districts embraced two approaches: they created or enhanced their own programs 
for preparing principals; and they forged closer working relationships with “partner programs” in nearby 
universities and other organizations. For four of the districts (Denver, Gwinnett County, Hillsborough 
County, and Prince George’s County), the district-run programs were especially critical parts of the district 
strategy for preparation, offering what district leaders saw as a particularly valuable capstone following the 
preparation that led to administrator certification. Charlotte-Mecklenburg and New York City maintained 
their position as assertive “consumers” of the graduates of outside programs, while New York City also en- 
larged district-run programs as key parts of its preparation portfolio. 
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The hiring process was an area in which district leaders expressed particular confidence that they had made 
critical improvements. A trusted policy instrument here was the repeated use of assessment as aspiring lead- 
ers made their way toward the ultimate stage of placement as a principal. Most districts installed additional 
selection gates along the way to hiring, such as selection into a talent pool from which hiring managers 
would choose new principals. New York City did not make this particular change, but it did introduce new 
assessments — aligned with standards — at the final stage before aspirants could apply for jobs. In all dis- 
tricts, then, hiring managers had access to additional information, systematically collected, about the spe- 
cific competencies of candidates. District leaders were enthusiastic about the power of assessment data, 
although their enthusiasm was tempered by their observation of two issues: the new rating procedures were 
time-consuming for central staff and ran the risk of creating burnout; and many recognized that interper- 
sonal leadership qualities were not well captured in the data systems. 

In reflecting on their districts’ ongoing work with evaluation and support systems, district leaders again 
hailed the value of standards and data. The same standards and competencies that shaped principal prepa- 
ration, job descriptions, and assessments for hiring could also be put to work in evaluating novice princi- 
pals’ strengths and weaknesses, and the resulting data could be captured and used for individual targeting 
of professional development and support. Supervision that blended evaluation and support was a preferred 
approach, and districts added staff who would provide mentoring or coaching to principals. 

In this component-by-component summary of the districts’ work, it is important to emphasize the extent to 
which strategic thinking underlay the districts’ actions and plans. Top leaders interviewed for this study 
saw how their work on preparation could support better hiring decisions; how ambiguities or gaps that 
came to light in principal evaluation could inform fine-tuning of standards documents; how principal super- 
visors could help refine the practical demonstrations and scoring rubrics used in hiring, based on their deep 
knowledge of leadership in the district; and other ways in which districts’ work and learning were con- 
nected throughout components of the initiative. 

District capacity for designing and managing a pipeline was growing and, along with it, district capacity to 
use the pipeline to support educational priorities. The Leader Tracking System, for example, brought a new 
kind of capacity for data-informed decisions that district leaders welcomed. All of the districts built their 
staff capacity to manage relationships with preferred external preparation programs, and five of them 
added to their in-house capacity for principal preparation. Capacity for succession planning was growing 
with the introduction of new organizational routines for this task. New positions for principal mentoring, 
coaching, and supervision brought more staff capacity to the functions of support and evaluation, as did 
the training in these functions offered by partner organizations. And, once they had these and other kinds 
of capacity in place, district leaders were eager to use them for strategic improvements in school leadership. 

As districts looked ahead, they focused increasingly on their assistant principals. Their data, like ours, re- 
vealed that most new principals came from the ranks of assistant principals. Therefore, the processes for 
preparing, selecting, evaluating, and coaching assistant principals were building blocks for the future of the 
district’s leadership. District leaders were attending not only to the individuals currently in those positions, 
sizing up their potential for advancement, but also to the systems that they had in place for developing and 
supporting assistant principals, recognizing that they had much work to do in improving those systems. 



The process of implementing the Principal Pipeline Initiative did not end in 2014. Districts’ work plans set 
further milestones to be reached before the grants expire in 2016. Just as important, district leaders contin- 
ued to revisit and refine the work that they had already done, seeking improvement in every component as 
they continued to learn from experience. 
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Appendix A: Survey Weighting and Analyses 


To determine post-stratification survey weights for each district, the inverse of the number of respondents 
from the district out of the total number of respondents to the survey was divided by six, the total number of 
districts (Exhibit A-l). For example, in 2014, there were 52 principal respondents in Charlotte-Mecklenburg 
and 541 total principal respondents from all six districts, resulting in a survey weight of 1.73 (i.e., 
(541/52)/6). The 2013 data from principals and assistant principals were weighted in a similar fashion. 


Exhibit A-1: 

Principal and assistant principal survey weights by district and year 


District 

2013 

2014 

Principal 

weight 

Assistant principal 
weight 

Principal 

weight 

Assistant 
principal weight 

Charlotte-Mecklenburg 

1.37 

1.12 

1.73 

1.26 

Denver 

1.05 

0.76 

1.33 

1.23 

Gwinnett County 

1.51 

0.62 

1.92 

0.77 

Hillsborough County 

1.07 

1.48 

2.15 

1.15 

New York City 

0.58 

N/A 

0.34 

0.54 

Prince George's County 

1.01 

2.80 

1.43 

2.80 


Exhibit reads: The survey weight for Charlotte-Mecklenburg principals in 2013 was 1.37; the survey weight for Charlotte-Mecklenburg assistant principals in 2013 was 1.12. The 
survey weight for Charlotte-Mecklenburg principals in 2014 was 1.73; the survey weight for Charlotte-Mecklenburg assistant principals in 2014 was 1.26. 

Note: Survey weights for 201 3 assistant principals were calculated out of five rather than six districts. The response rate in New York City among assistant principals was low, so 
those assistant principal responses were excluded from analysis in 2013. 


The main method of analysis for this report was to analyze cross-sectional data from 2013 and 2014. The 
2013 data included principals and assistant principals who started in their position from 2010 to 2013, and 
the 2014 data included principals and assistant principals who started their position from 2011 to 2014. 
Some of the cross-tabulations in this report examine the differences between principals and assistant princi- 
pals who started their positions at different points in time. In creating cohorts of principals, the 2010-11 co- 
hort includes any principal or assistant principal who started between March 2010 and February 2011; the 
2011-12 cohort includes any principal or assistant principal who started between March 2011 and February 
2012; the 2012-13 cohort includes any principal or assistant principal who started between March 2012 and 
February 2013; and the 2013-14 cohort includes any principal or assistant principal who started between 
March 2013 and February 2014. For comparisons that required responses from the 2010-11 cohort, 2013 
survey weights were applied to their data; 2014 survey weights were applied for all other cohorts. 
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Many comparisons in the report are presented along with information about whether the difference between 
estimated percentages is statistically significant. One method of determining the statistical significance of the 
difference between two percentages is to compare the confidence intervals of the two percentages. 
Confidence intervals provide information about the accuracy of the estimated percentages. If the confidence 
intervals for two percentages do not overlap, then the difference is deemed statistically significant. 
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Appendix B: An Exploratory Comparative Analysis 


Our principal survey affords an opportunity for an exploratory comparison of principals’ reports on their 
preparation with the findings of an important prior study (Darling-Hammond et ah, 2007), which helped in- 
form the design of the Principal Pipeline Initiative. The prior study surveyed two groups of respondents: 
graduates of the programs highlighted as exemplars and a national comparison group of principals. Our sur- 
vey replicates items from that study, allowing a comparison of the results. 

Caveats are in order for these comparisons. One relates to the time that has elapsed between the prior and 
current research. A national survey conducted in 2013 or 2014 might well be expected to yield results differ- 
ent from those of respondents surveyed in the previous decade. It is likely that the preparation programs for 
principals hired recently in the Principal Pipeline Initiative districts reflected current trends in school leader- 
ship practice, such as newly intensified expectations for data use. Moreover, they might reflect attention to 
the research on principal preparation that had not yet been reported a decade ago. However, the extent to 
which preparation programs approach or exceed the standard held up as exemplary in the prior research is 
of interest, and we report on it annually in this evaluation. 

With respect to assessments of program content and instructional approaches, and self-reported perceptions 
of their preparation for specific aspects of leadership, we found that the reports of novice principals in these 
districts were fairly similar to those of graduates of the exemplary programs studied by Darling-Hammond 
and colleagues, and somewhat more favorable than those of that study’s national sample. In comparisons 
where the effect size exceeds .50, we interpret the difference as at least moderate in size and therefore likely 
to be meaningful. Many of the differences that appear in these exhibits, although statistically significant, 
show small effect sizes; in these cases we do not interpret the apparent differences as being meaningful. 

Thus, for example, principals in our 2014 survey reported much greater emphasis on leadership for school 
improvement in their preservice than did Darling-Hammond’s national sample (d=. 99) (Exhibit B-2). On the 
other hand, we do not view the apparent difference in emphasis on leadership for school improvement be- 
tween the principals we surveyed and the graduates of Darling-Hammond’s exemplary programs as mean- 
ingful; this is because, although the difference was statistically significant, the effect size was small (d--. 15). 

With those caveats and suggestions for interpretation, we show in Exhibits B-l through B-6 the comparisons 
between our survey results and those of the prior study. 
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Exhibit B-1: Principal perceptions of preservice content, 2013 


To what extent were the 
following qualities/practices 
true of your formal leadership 
preparation training? 

PPI Evaluation 
Surveys in 2013 

Darling-Hammond et al. data reported in 2007 

Principal 

( 0 = 326 ) 

Exemplary Program 

(N= 242 ) 

National Comparison 

(N= 62 g) 

Mean 

Effect Size 

Mean 

Effect Size 

a. Program content emphasized 
leadership for school 
improvement 

4.27 

4.49** 

-.26 

3.63** 

.77 

b. Program content emphasized 
instructional leadership 

4.27 

4.58** 

-.37 

4.13* 

.17 

c. Program gave me a strong 
orientation to the principalship 
as a career 

4.06 

4.39** 

-.39 

3.73** 

.39 
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Exhibit reads: Principal responses averaged 4.27 on the extent to which program content emphasized leadership for school improvement. Darling-Hammond et al. exemplary programs responses averaged 
4.49 and the national comparison responses averaged 4.39. Response means in this table are on a five-point scale in which 1 equals “not at all” and 5 equals “to a great extent.” 

Note: T-tests were conducted to determine statistical significance; *p<.0 5 and **p<.01 . Effect sizes were calculated using Cohen’s d. We suggest interpreting effect sizes using d=+/-. 50 as a minimum threshold 
for interpreting results as having at least a moderate effect. 

Source: Principal Sun/ey for “Evaluation of the Principal Pipeline Initiative,” 201 3; Principal suiveys reported in Darling-Hammond, L, LaPointe, M., Meyerson, D., Orr. M. I, & Cohen, C. (2007). Preparing school 
leaders fora changing world: Lessons from exemplary leadership development programs. Stanford, CA: Stanford University, Stanford Educational Leadership Institute. 


Exhibit B-2: Principal perceptions of preservice content, 2014 


To what extent were the 
following qualities/practices 
true of your formal leadership 
preparation training? 


PPI Evaluation 
Surveys in 2014 

Darling-Hammond et al. data reported in 2007 

Principal 

(n=472) 

Exemplary Program 

(N= 242 ) 

National Comparison 

(N= 62 g) 

Mean Effect Size 

Mean Effect Size 


a. Program content emphasized 
instructional leadership 

4.43 

4.58** 

-.20 

4.13** 

.39 

b. Program content emphasized 
leadership for school 
improvement 

4.38 

4.49** 

-.15 

3.63** 

.99 

c. Program gave me a strong 
orientation to the principalship 
as a career 

4.20 

4.39** 

-.23 

3.73** 

.57 


Exhibit reads: Principal responses averaged 4.43 on the extent to which program content emphasized instructional leadership. Darling-Hammond et al. exemplaiy programs responses averaged 4.58 and the 
national comparison responses averaged 4.13. Response means in this table are on a five-point scale in which 1 equals “not at all” and 5 equals “to a great extent.” 

Note: T-tests were conducted to determine statistical significance; *p<.05 and **p<.01. Effect sizes were calculated using Cohen’s d. We suggest interpreting effect sizes using d=+/-. 50 as a minimum threshold 
for interpreting results as having at least a moderate effect. 

Source: Principal Survey for “Evaluation ofthe Principal Pipeline Initiative,” 2014; Principal surveys reported in Darling-Hammond, L., LaPointe, M., Meyerson, D., Orr. M.T., 8i Cohen, C. (2007). Preparing school 
leaders fora changing world: Lessons from exemplary leadership development programs. Stanford, CA: Stanford University, Stanford Educational Leadership Institute. 





Exhibit B-3: Principal perceptions of preservice coursework, 2013 


To what extent were the 
following leadership 
practices/instructional 
strategies part of your 
coursework in your leadership 
preparation training? 

PPI Evaluation 
Surveys in 2013 

Darling-Hammond et al. data reported in 2007 

Principal 

(n= 326 ) 

Exemplary Program 

(N= 242 ) 

National Comparison 

(N= 62 p) 

Mean Effect Size 

Mean Effect Size 


a. Analysis and discussion of 
case studies 

3.96 

4.39** 

-.38 

3.74** 

.19 

b. Analysis and discussion of 
field-based problems/problem- 
based learning approaches 

3.94 

4.29** 

-.30 

3.47** 

.41 

c. A portfolio demonstrating your 
learning and accomplishments 

3.88 

4.36** 

-.38 

2.81** 

.84 

d. Field-based projects in which 
you applied ideas in the field 

3.84 

4.22** 

-.31 

3.37** 

.39 

e. Action research or 
inquiry projects 

3.66 

4.00** 

-.28 

3.34** 

.26 


Exhibit reads: Principal responses averaged 3.96 on to what extent analysis and discussion of case studies was a part of their coursework. Darling-Hammond et al. exemplary programs responses averaged a 
4.39 and the national comparison responses averaged a 3.74. Response means in this table are on a five-point scale in which 1 equals “not at all” and 5 equals “to a great extent.” 

Note: T-tests were conducted to determine statistical significance; *p<.0 5 and **p<.01. Effect sizes were calculated using Cohen’s d. We suggest interpreting effect sizes using d=+/-. 50 as a minimum threshold 
for interpreting results as having at least a moderate effect 

Source: Principal Survey for “Evaluation ofthe Principal Pipeline Initiative,” 2013; Principal surveys reported in Darling-Hammond, L, LaPointe, M., Meyerson, D., Orr. M.T., &Cohen, C. (2007). Preparing school 
leaders fora changing world: Lessons from exemplary leadership development programs. Stanford, CA: Stanford University, Stanford Educational Leadership Institute. 
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Exhibit B-4: Principal perceptions of preservice coursework, 2014 


To what extent were the 
following leadership 
practices/instructional 
strategies part of your 
coursework in your leadership 
preparation training? 


PPI Evaluation 
Surveys in 2014 

Darling-Hammond et al. data reported in 2007 

Principal 

( 11 = 472 ) 

Exemplary Program 

(N= 242 ) 

National Comparison 

(H= 6 ig) 

Mean Effect Size 

Mean Effect Size 


a. Analysis and discussion of 
case studies 

4.15 

4.39** 

-.25 

3.74** 

.42 

b. Analysis and discussion of 
field-based problems/problem- 
based learning approaches 

4.12 

4.29** 

-.18 

3.47** 

.67 

c. A portfolio demonstrating your 
learning and accomplishments 

3.98 

4.36** 

-.31 

2.81** 

.95 

d. Field-based projects in which 
you applied ideas in the field 

3.97 

4.22** 

-.23 

3.37** 

.55 

e. Action research or 
inquiry projects 

3.86 

4.00** 

-.12 

3.34** 

.46 
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Exhibit reads: Principal responses averaged 4.1 5 on to what extent analysis and discussion of case studies was a part of their coursework. Darling-Hammond et al. exemplary programs responses averaged a 
4.39 and the national comparison responses averaged a 3.74. Response means in this table are on a five-point scale in which 1 equals “not at all” and 5 equals “to a great extent.” 

Note: T-tests were conducted to determine statistical significance; *p<.0 5 and **p<.01 . Effect sizes were calculated using Cohen’s d. We suggest interpreting effect sizes using d=+/-. 50 as a minimum threshold 
for interpreting results as having at least a moderate effect. 

Source: Principal Survey for “Evaluation of the Principal Pipeline Initiative,” 2014; Principal suiveys reported in Darling-Hammond, L, LaPointe, M., Meyerson, D., Orr. M. I, & Cohen, C. (2007). Preparing school 
leaders fora changing world: Lessons from exemplary leadership development programs. Stanford, CA Stanford University, Stanford Educational Leadership Institute. 




Exhibit B-5: Principal perceptions of their preparedness, 2013 


How effectively did your leadership 
preparation training prepare you 
to do each of the following: 

PPI Evaluation 
Surveys in 2013 

Darling-Hammond et al. data reported in 2007 

Principal 

(n= 326 ) 

Exemplary Program 

(N= 242 ) 

National Comparison 

(N= 62 ff) 

Mean 

Effect Size 

Mean 

Effect Size 

a. Engage in self-improvement 
and continuous learning 

3.38 

3.58** 

-.28 

291 ** 

.64 

b. Use data to monitor school progress, 
identify problems and propose solutions 

3.18 

3.31** 

-.15 

2.47** 

.82 

c. Create a collaborative learning 
organization 

3.17 

3.34** 

-.22 

2.69** 

.63 

d. Engage staff in a decision making process 
about school curriculum and policies 

3.13 

3.22* 

-.11 

2.70** 

.52 

e. Evaluate teachers and provide 

instructional feedback to support their 3.10 

improvement 

3.36** 

-.31 

2.82** 

.33 

f. Lead a well-informed, planned change 
process for a school 

3.10 

3.22** 

-.15 

2.59** 

.63 

g. Engage in comprehensive planning for 
school improvement 

3.09 

3.22** 

-.15 

2.58** 

.63 

h. Create a coherent educational program 
across the school 

2.96 

3.22** 

-.32 

2.63** 

.41 

i. Redesign school organizations to enhance 
productive teaching and learning 

2.92 

3.06** 

-.16 

2.46** 

.55 

j. Design professional development that 
builds teachers' knowledge and skills 

2.87 

3.25** 

-.44 

2.50** 

.42 

k. Understand how different students learn 
and how to teach them successfully 

2.70 

3.03** 

-.38 

2.56** 

.16 

1. Evaluate curriculum materials for their 
usefulness in supporting learning 

2.56 

2.92** 

-.39 

2.49 

.08 


Exhibit reads: Principal responses averaged 3.38 on how prepared they were to engage in self-improvement and continuous learning. Darling-Hammond et al. exemplary programs responses averaged 3.58 
and the national comparison responses averaged 2.91. Responses on this table are on a four-point scale in which 1 equals “not at all” and 4 equals “very well.” 

Note: Response means from the Darling-Hammond et al. survey were converted from a five-point scale to a four-point scale to allow for comparisons. 

Note: T-tests were conducted to determine statistical significance; *p<.05 and **p<.01 . Effect sizes were calculated using Cohen’s d. We suggest interpreting effect sizes using d=+/-. 50 as a minimum threshold 
for interpreting results as having at least a moderate effect. 

Source: Principal Survey for “Evaluation ofthe Principal Pipeline Initiative,” 2013; Principal surveys reported in Darling-Hammond, L, LaPointe, M., Meyerson, D., Orr. M.T., 8i Cohen, C. (2007). Preparing school 
leaders fora changing world: Lessons from exemplary leadership development programs. Stanford, CA: Stanford University, Stanford Educational Leadership Institute. 
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Exhibit B-6: Principal perceptions of their preparedness, 2014 


How effectively did your leadership 
preparation training prepare you 
to do each of the following: 

PPI Evaluation 
Surveys in 2014 

Darling-Hammond et al. data reported in 2007 

Principal 

(n=476) 

Exemplary Program 

(N= 242 ) 

National Comparison 

(N= 62 g) 

Mean Effect Size 

Mean Effect Size 


Engage in self-improvement 
and continuous learning 

3.36 

3.58** 

-.31 

2 91** 

.62 

Create a collaborative learning 
organization 

3.24 

3.34** 

-.12 

2.69** 

.71 

Evaluate teachers and provide 
instructional feedback to support their 
improvement 

3.24 

3.36** 

-.16 

2.82** 

.54 

Use data to monitor school progress, 
identify problems and propose solutions 

3.16 

3.31** 

-.18 

2.47** 

.83 

Lead a well-informed, planned change 
process for a school 

3.16 

3.22 

-.08 

2.59** 

.73 

Engage in comprehensive planning for 
school improvement 

3.14 

3.22* 

-.09 

2.58** 

.68 

Engage staff in a decision making process 
about school curriculum and policies 

3.11 

3.22** 

-.14 

2.70** 

.52 

Create a coherent educational program 
across the school 

3.05 

3.22** 

-.21 

2.63** 

.53 

Design professional development that 
builds teachers' knowledge and skills 

2.96 

3.25** 

-.32 

2.50** 

.51 

Redesign school organizations to enhance 
productive teaching and learning 

2.93 

3.06** 

-.15 

2.46** 

.58 

Understand how different students learn 
and how to teach them successfully 

2.75 

3.03** 

-.32 

2.56** 

.22 

Evaluate curriculum materials for their 
usefulness in supporting learning 

2.67 

2.92** 

-.28 

2 49 ** 

.20 


Exhibit reads: Principal responses averaged 3.36 on how prepared they were to engage in self-improvement and continuous learning. Darling-Hammond et al. exemplary programs responses averaged 3.58 
and the national comparison responses averaged 2.91. Responses on this table are on a four-point scale in which 1 equals “not at all” and 4 equals “very well.” 

Note: Response means from the Darling-Hammond et al. survey were converted from a five-point scale to a four-point scale to allow for comparisons. 

Note: T-tests were conducted to determine statistical significance; *p<.05 and **p<.01. Effect sizes were calculated using Cohen’s d. We suggest interpreting effect sizes using d=+/-. 50 as a minimum threshold 
for interpreting results as having at least a moderate effect. 

Source: Principal Survey for “Evaluation ofthe Principal Pipeline Initiative,” 2014; Principal surveys reported in Darling-Hammond, L, LaPointe, M., Meyerson, D., Orr. M. I, & Cohen, C. (2007). Preparing school 
leaders fora changing world: Lessons from exemplary leadership development programs. Stanford, CA: Stanford University, Stanford Educational Leadership Institute. 
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